model

package
v0.0.0-...-c912a3c Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 27, 2024 License: Apache-2.0 Imports: 66 Imported by: 0

Documentation

Overview

Package model contains datastore model definitions.

Package model contains model definitions for Auth Service.

This file contains functionality related to the "direct push" method of AuthDB replication.

Index

Constants

View Source 
const (
	DryRunAPIChangesEnvVar    = "DRY_RUN_API_CHANGES"
	DryRunCronConfigEnvVar    = "DRY_RUN_CRON_CONFIG"
	DryRunCronRealmsEnvVar    = "DRY_RUN_CRON_REALMS"
	DryRunCronStaleAuthEnvVar = "DRY_RUN_CRON_STALE_AUTH"
	DryRunTQChangelogEnvVar   = "DRY_RUN_TQ_CHANGELOG"
	DryRunTQReplicationEnvVar = "DRY_RUN_TQ_REPLICATION"
	EnableGroupImportsEnvVar  = "ENABLE_GROUP_IMPORTS"
)

Names of enviroment variables which control component functionality while transitioning from Auth Service v1 (Python) to Auth Service v2 (Go).

Each environment variable should be either "true" or "false" (case insensitive).

View Source 
const (
	// AdminGroup defines a group whose members are allowed to create new groups.
	AdminGroup = "administrators"

	// TrustedServicesGroup defines a group whose members are allowed to
	// subscribe to group change notifications and fetch all groups at
	// once.
	TrustedServicesGroup = "auth-trusted-services"

	// Max size of AuthDBShard.Blob.
	MaxShardSize = 900 * 1024 // 900 kB.

	// The prefix to use for datastore entities' Kind field when in dry
	// run mode.
	DryRunPrefix = "V2"
)
View Source 
const (
	// Object ACLs can have at most 100 entries. We limit them to 80 to
	// have some breathing room before the hard limit is reached. When
	// this happens, either some old services should be deauthorized or
	// GCS ACL management should be reimplemented in some different way.
	//
	// See https://cloud.google.com/storage/quotas.
	MaxReaders = 80

	// Limit for an AuthDBReader email, because the maximum size of an
	// indexed string property's UTF-8 encoding in Datastore is 1500 B.
	//
	// See https://cloud.google.com/datastore/docs/concepts/limits.
	MaxReaderEmailLength = 375
)
View Source 
const (
	// Defines version of Auth API.
	//
	// Should be increased on any API or protocol changes.
	AuthAPIVersion = "2.0.0"
)
View Source 
const RealmsAPIVersion = 1

RealmsAPIVersion is the currently acceptable version of Realms API. See api_version in realms.proto.

Variables

View Source 
var (
	// ErrConcurrentAuthDBUpdate signifies the AuthDB was modified in a concurrent transaction.
	ErrConcurrentAuthDBUpdate = errors.New("AuthDB changed between transactions")
	// ErrImporterNotConfigured is returned if there is no importer config.
	ErrImporterNotConfigured = errors.New("no importer config")
	// ErrInvalidTarball is returned if the tarball data is invalid.
	ErrInvalidTarball = errors.New("invalid tarball data")
	// ErrInvalidTarballName is returned if the tarball name is invalid.
	ErrInvalidTarballName = errors.New("invalid tarball name")
	// ErrUnauthorizedUploader is returned if the caller is not authorized to upload a given tarball.
	ErrUnauthorizedUploader = errors.New("unauthorized tarball uploader")
)
View Source 
var (
	// ErrAlreadyExists is returned when an entity already exists.
	ErrAlreadyExists = stderrors.New("entity already exists")
	// ErrPermissionDenied is returned when the user does not have permission for
	// the requested entity operation.
	ErrPermissionDenied = stderrors.New("permission denied")
	// ErrInvalidArgument is a generic error returned when an argument is invalid.
	ErrInvalidArgument = stderrors.New("invalid argument")
	// ErrInvalidName is returned when a supplied entity name is invalid.
	ErrInvalidName = stderrors.New("invalid entity name")
	// ErrInvalidReference is returned when a referenced entity name is invalid.
	ErrInvalidReference = stderrors.New("some referenced groups don't exist")
	// ErrInvalidIdentity is returned when a referenced identity or glob is invalid.
	ErrInvalidIdentity = stderrors.New("invalid identity")
	// ErrConcurrentModification is returned when an entity is modified by two concurrent operations.
	ErrConcurrentModification = stderrors.New("concurrent modification")
	// ErrReferencedEntity is returned when an entity cannot be deleted because it is referenced elsewhere.
	ErrReferencedEntity = stderrors.New("cannot delete referenced entity")
	// ErrCyclicDependency is returned when an update would create a cyclic dependency.
	ErrCyclicDependency = stderrors.New("groups can't have cyclic dependencies")
)
View Source 
var (
	// ErrInvalidTarget is returned if the target for a change log listing
	// query is invalid.
	ErrInvalidTarget = errors.New("invalid target")
)
View Source 
var (
	FatalReplicaUpdateError = errors.New("fatal replica update error")
)

Functions

func AuthorizeReader 

func AuthorizeReader(ctx context.Context, email string) error

AuthorizeReader records the given email as an AuthDBReader, then updates Auth Service's Google Storage object ACLs.

func CanCallerModify 

func CanCallerModify(ctx context.Context, group *AuthGroup) (bool, error)

CanCallerModify returns whether the current identity can modify the given group.

func CanCallerViewMembers 

func CanCallerViewMembers(ctx context.Context, group *AuthGroup) (bool, error)

CanCallerViewMembers returns whether the current identity can view the members of the given group.

func ChangeID 

func ChangeID(ctx context.Context, change *AuthDBChange) (string, error)

ChangeID returns the ID of an AuthDBChange entity based on its properties.

Returns error when change.Target is invalid (doesn't contain '$' or contains '!').

func ChangeLogRevisionKey 

func ChangeLogRevisionKey(ctx context.Context, authDBRev int64, dryRun bool) *datastore.Key

ChangeLogRevisionKey returns the key of entity subgroup that keeps AuthDB change log for a revision.

func ChangeLogRootKey 

func ChangeLogRootKey(ctx context.Context, dryRun bool) *datastore.Key

ChangeLogRootKey returns the root key of an entity group with change log.

func CompareV2Entities 

func CompareV2Entities(ctx context.Context) error

CompareV2Entities compares the entities generated by Auth Service v1 (Python) to those generated by Auth Service v2 (Go) for the latest AuthDB revision. The comparison result is logged.

Returns an annotated error if one occurred.

func DeauthorizeReader 

func DeauthorizeReader(ctx context.Context, email string) error

DeauthorizeReader removes the given email as an AuthDBReader, then updates Auth Service's Google Storage object ACLs.

func DeleteAuthGroup 

func DeleteAuthGroup(ctx context.Context, groupName string, etag string, historicalComment string, dryRun bool) error

DeleteAuthGroup deletes the specified AuthGroup.

Possible errors: 

datastore.ErrNoSuchEntity if the specified group does not exist.
ErrPermissionDenied if the caller is not allowed to delete the group.
ErrConcurrentModification if the provided etag is not up-to-date.
ErrReferencedEntity if the group is referenced by another group.
Annotated error for other errors.

func EnqueueProcessChangeTask 

func EnqueueProcessChangeTask(ctx context.Context, authdbrev int64) error

EnqueueProcessChangeTask adds a ProcessChangeTask task to the cloud task queue.

func EnqueueReplicationTask 

func EnqueueReplicationTask(ctx context.Context, authdbrev int64) error

EnqueueReplicationTask adds a ReplicationTask task to the cloud task queue.

func FromStorableRealms 

func FromStorableRealms(blob []byte) (*protocol.Realms, error)

func GetAuthorizedEmails 

func GetAuthorizedEmails(ctx context.Context) (stringset.Set, error)

GetAuthorizedEmails returns the emails of all AuthDBReaders.

func HistoricalRevisionKey 

func HistoricalRevisionKey(ctx context.Context, authDBRev int64) *datastore.Key

HistoricalRevisionKey gets the key for an entity subgroup that holds changes done in a concrete revision.

func IngestTarball 

func IngestTarball(ctx context.Context, name string, content io.Reader) ([]string, int64, error)

IngestTarball handles upload of tarball's specified in 'tarball_upload' config entries. expected to be called in an auth context of the upload PUT request.

returns 

[]string - list of modified groups
int64 - authDBRevision
error
	ErrImporterNotConfigured if no importer config
	ErrUnauthorizedUploader if caller is not an authorized uploader
	ErrInvalidTarballName if tarball name is empty
	ErrInvalidTarballName if entry not found in tarball upload config
	ErrInvalidTarball if bad tarball structure
	error from importing the tarball, if one occurs

func IsAuthorizedReader 

func IsAuthorizedReader(ctx context.Context, email string) (bool, error)

IsAuthorizedReader returns whether the given email is authorized to read the AuthDB from Google Storage.

func IsExternalAuthGroupName 

func IsExternalAuthGroupName(name string) bool

IsExternalAuthGroupName checks if the given name is a valid external auth group name. This also implies that the auth group is not editable.

func MergeRealms 

func MergeRealms(
	ctx context.Context,
	realmsGlobals *AuthRealmsGlobals,
	allAuthProjectRealms []*AuthProjectRealms,
	useV1Perms bool) (*protocol.Realms, error)

MergeRealms merges all the project realms into one realms definition, using the permissions in the given AuthRealmsGlobals as an authoritative source of valid permissions.

If some realm uses a permission not in the list, it will be silently dropped from the bindings. This can potentially happen due to the asynchronous nature of realms config updates (e.g. a role change that deletes some permissions can be committed into the AuthDB before realms are reevaluated). Eventually, the state should converge to be 100% consistent.

Returns a single Realms object, which represents all projects' realms if there were no issues when merging, otherwise nil and an annotated error.

An error may occur when: * unmarshalling project realms; * merging conditions into a single set; * processing a single realm that doesn't have the project ID as a prefix; * relabelling condition indices relative to the new set of all conditions.

func ParseDryRunEnvVar 

func ParseDryRunEnvVar(envVar string) bool

ParseDryRunEnvVar parses the dry run flag from the given environment variable, defaulting to true.

func ParseEnableEnvVar 

func ParseEnableEnvVar(envVar string) bool

ParseEnableEnvVar parses the enable flag from the given environment variable, defaulting to false.

func PreviousHistoricalRevisionKey 

func PreviousHistoricalRevisionKey(ctx context.Context, pm datastore.PropertyMap) *datastore.Key

PreviousHistoricalRevisionKey gets the key for the previous history entity of the given historical entity PropertyMap.

func RealmsConfigCronHandler 

func RealmsConfigCronHandler(dryRun bool) func(context.Context) error

func RegisterChangeHandler 

func RegisterChangeHandler(dryRun bool)

RegisterChangeHandler registers the handler for changelog generation tasks.

func RegisterReplicationHandler 

func RegisterReplicationHandler(dryRunReplication, useV1Perms bool)

RegisterReplicationHandler registers the handler for replication tasks.

func ReplicasRootKey 

func ReplicasRootKey(ctx context.Context) *datastore.Key

ReplicasRootKey is the root key for AuthReplicaState entities. The entity itself doesn't exist.

func ReplicateToReplica 

func ReplicateToReplica(ctx context.Context, r *AuthReplicaState,
	authDBRev int64, authDBBlob []byte, keyName, encodedSig string) error

ReplicateToReplica pushes a revision of the AuthDB to the given replica, then updates the last known state of the replica.

Returns an error if: - the push to the replica cannot be considered successful; or - there was an error updating the last known state of the replica.

func ReplicatedAuthDBRefresher 

func ReplicatedAuthDBRefresher(ctx context.Context) error

ReplicatedAuthDBRefresher triggers AuthDB replication if it hasn't been done recently, as defined by maxAuthDBAge.

Called periodically as a cron job. If it detects that the last AuthDB revision was produced awhile ago, bumps AuthDB revision number and triggers replication (actual contents of AuthDB is not changed).

This is important to make sure the AuthDB replication configuration doesn't rot and that the exported AuthDB blob has a relatively fresh signature.

func RevokeStaleReaderAccess 

func RevokeStaleReaderAccess(ctx context.Context, trustedGroup string, dryRun bool) error

RevokeStaleReaderAccess deletes any AuthDBReaders that are no longer in the given trusted group, then updates Auth Service's Google Storage object ACLs.

func RootKey 

func RootKey(ctx context.Context) *datastore.Key

RootKey gets the root key of the entity group with all AuthDB entities.

func ServiceConfigCronHandler 

func ServiceConfigCronHandler(dryRun bool) func(context.Context) error

func StaleAuthorizationCronHandler 

func StaleAuthorizationCronHandler(dryRun bool) func(context.Context) error

func StoreAuthDBSnapshot 

func StoreAuthDBSnapshot(ctx context.Context, replicationState *AuthReplicationState, authDBBlob []byte, dryRun bool) (err error)

StoreAuthDBSnapshot stores the AuthDB blob (serialized proto) into Datastore.

Args:

  • replicationState: the AuthReplicationState corresponding to the given authDBBlob.
  • authDBBlob: serialized protocol.ReplicationPushRequest message (has AuthDB inside).

func ToStorableRealms 

func ToStorableRealms(realms *protocol.Realms) ([]byte, error)

Types

type AuthDBChange 

type AuthDBChange struct {
	Kind   string         `gae:"$kind,AuthDBChange"`
	ID     string         `gae:"$id"`
	Parent *datastore.Key `gae:"$parent"`

	Class []string `gae:"class"` // A list of "class" names giving the NDB Polymodel hierarchy

	// Fields common across all change types.
	ChangeType     ChangeType `gae:"change_type" json:"change_type"` // What kind of a change this is (see Change*)
	Target         string     `gae:"target" json:"target"`           // Entity (or subentity) that was changed: kind$id[$subid] (subid is optional).
	AuthDBRev      int64      `gae:"auth_db_rev" json:"auth_db_rev"` // AuthDB revision at which the change was made.
	Who            string     `gae:"who" json:"who"`                 // Who made the change.
	When           time.Time  `gae:"when" json:"when"`               // When the change was made.
	Comment        string     `gae:"comment" json:"comment"`         // Comment passed to record_revision or record_deletion.
	AppVersion     string     `gae:"app_version" json:"app_version"` // GAE application version at which the change was made.
	Description    string     `gae:"description,noindex" json:"description"`
	OldDescription string     `gae:"old_description,noindex" json:"old_description"`

	// Fields specific to AuthDBGroupChange.
	Owners    string   `gae:"owners" json:"owners"`         // Valid for ChangeGroupCreated and ChangeGroupOwnersChanged.
	OldOwners string   `gae:"old_owners" json:"old_owners"` // Valid for ChangeGroupOwnersChanged and ChangeGroupDeleted.
	Members   []string `gae:"members" json:"members"`       // Valid for ChangeGroupMembersAdded and ChangeGroupMembersRemoved.
	Globs     []string `gae:"globs" json:"globs"`           // Valid for ChangeGroupGlobsAdded and ChangeGroupGlobsRemoved.
	Nested    []string `gae:"nested" json:"nested"`         // Valid for ChangeGroupNestedAdded and ChangeGroupNestedRemoved.

	// Fields specific to AuthDBIPAllowlistChange.
	Subnets []string `gae:"subnets" json:"subnets"` // Valid for ChangeIPWLSubnetsAdded and ChangeIPWLSubnetsRemoved.

	// Fields specific to AuthDBIPAllowlistAssignmentChange.
	Identity    string `gae:"identity"` // Valid for ChangeIPWLAssignSet and ChangeIPWLAssignUnset.
	IPAllowlist string `gae:"ip_whitelist"`

	// Fields specific to AuthDBConfigChange.
	OauthClientID            string   `gae:"oauth_client_id" json:"oauth_client_id"`                         // Valid for ChangeConfOauthClientChanged.
	OauthClientSecret        string   `gae:"oauth_client_secret" json:"oauth_client_secret"`                 // Valid for ChangeConfOauthClientChanged.
	OauthAdditionalClientIDs []string `gae:"oauth_additional_client_ids" json:"oauth_additional_client_ids"` // Valid for ChangeConfClientIDsAdded and ChangeConfClientIDsRemoved.
	TokenServerURLOld        string   `gae:"token_server_url_old" json:"token_server_url_old"`               // Valid for ChangeConfTokenServerURLChanged.
	TokenServerURLNew        string   `gae:"token_server_url_new" json:"token_server_url_new"`               // Valid for ChangeConfTokenServerURLChanged.
	SecurityConfigOld        []byte   `gae:"security_config_old,noindex" json:"security_config_old"`         // Valid for ChangeConfSecurityConfigChanged.
	SecurityConfigNew        []byte   `gae:"security_config_new,noindex" json:"security_config_new"`         // Valid for ChangeConfSecurityConfigChanged.

	// Fields specific to AuthRealmsGlobalsChange.
	PermissionsAdded   []string `gae:"permissions_added" json:"permissions_added"`
	PermissionsChanged []string `gae:"permissions_changed" json:"permissions_changed"`
	PermissionsRemoved []string `gae:"permissions_removed" json:"permissions_removed"`

	// Fields specific to AuthProjectRealmsChange.
	ConfigRevOld string `gae:"config_rev_old,noindex"`
	ConfigRevNew string `gae:"config_rev_new,noindex"`
	PermsRevOld  string `gae:"perms_rev_old,noindex"`
	PermsRevNew  string `gae:"perms_rev_new,noindex"`
}

AuthDBChange is the base (embedded) struct for change log entries. Has a change type and a bunch of common change properties (like who and when made the change). Change type order is important, it is used in UI when sorting changes introduces by single AuthDB commit.

Change types represent minimal indivisible changes. Large AuthDB change is usually represented by (unordered) set of indivisible changes. For example, an act of creation of a new AuthDB group produces following changes: 

ChangeGroupCreated
ChangeGroupMembersAdded
ChangeGroupGlobsAdded
ChangeGroupNestedAdded

They are unordered, but UI sorts them based on change type integer. Thus ChangeGroupCreated appears on top: it's represented by the smallest integer.

Entity id has following format: 

<original_entity_kind>$<original_id>[$<subentity_id>]!<change_type>

where: 

original_entity_kind: a kind of modified AuthDB entity (e.g. 'AuthGroup')
original_id: ID of modified AuthDB entity (e.g. 'Group name')
subentity_id: optional identified of modified part of the entity, used for
  IP allowlist assignments entity (since it's just one big singleton).
change_type: integer Change* (see below), e.g. '1100'.

Such key structure makes 'diff_entity_by_key' operation idempotent. A hash of entity body could have been used too, but having readable (and sortable) keys are nice.

Parent entity is AuthDBRevision(ChangeLogRevisionKey).

Note: '$' and '!' are not likely to appear in entity names since they are forbidden in AuthDB names. Code here also asserts this.

func GetAllAuthDBChange 

func GetAllAuthDBChange(ctx context.Context, target string, authDBRev int64, pageSize int32, pageToken string) (changes []*AuthDBChange, nextPageToken string, err error)

GetAllAuthDBChange returns all the AuthDBChange entities with given target and authDBRev. If target is an empty string/authDBRev equals to 0, no target/authDBRev is specified.

Returns an annotated error.

func (*AuthDBChange) ToProto 

func (change *AuthDBChange) ToProto() *rpcpb.AuthDBChange

type AuthDBLogRev 

type AuthDBLogRev struct {
	Kind   string         `gae:"$kind,AuthDBLogRev"`
	ID     int64          `gae:"$id"` // The AuthDB revision.
	Parent *datastore.Key `gae:"$parent"`

	When       time.Time `gae:"when" json:"when"`               // When the changes were processed.
	AppVersion string    `gae:"app_version" json:"app_version"` // GAE application version that processed the change.
}

AuthDBLogRev is used to record that the changelog was generated for an AuthDB revision.

type AuthDBReader 

type AuthDBReader struct {
	// Account that should be able to read AuthDB Google Storage dump.
	//
	// These are accounts that have explicitly requested access to the
	// AuthDB via API call to
	// `/auth_service/api/v1/authdb/subscription/authorization`.
	//
	// They all belong to the `auth-trusted-services` group.
	// Note that we can't just authorize all members of
	// `auth-trusted-services` since we can't enumerate them
	// (for example, there's no way to enumerate glob entries like
	// *@example.com).
	Kind   string         `gae:"$kind,AuthDBReader"`
	Parent *datastore.Key `gae:"$parent"`

	// The account email.
	ID string `gae:"$id"`

	//AuthorizedTS is the time this account was last authorized.
	AuthorizedTS time.Time `gae:"authorized_at,noindex"`
}

type AuthDBShard 

type AuthDBShard struct {
	Kind string `gae:"$kind,AuthDBShard"`
	ID   string `gae:"$id"`

	// Blob represents a sharded part of the full AuthDB blob.
	Blob []byte `gae:"blob,noindex"`
}

AuthDBShard is a shard of deflated serialized ReplicationPushRequest. Root entity. ID is "<auth_db_revision:<blob hash>

type AuthDBSnapshot 

type AuthDBSnapshot struct {
	Kind string `gae:"$kind,AuthDBSnapshot"`
	ID   int64  `gae:"$id"`

	// AuthDBDeflated is the deflated serialized ReplicationPushRequest proto message.
	AuthDBDeflated []byte `gae:"auth_db_deflated,noindex"`

	// ShardIDs is a list of shard IDs if sharded or empty if auth_db_deflated should be used.
	ShardIDs []string `gae:"shard_ids,noindex"`

	// AuthDBSha256 is the SHA256 hex digest of auth_db (before compression).
	AuthDBSha256 string `gae:"auth_db_sha256,noindex"`

	// CreatedTS is when this snapshot was created.
	CreatedTS time.Time `gae:"created_ts"`
}

AuthDBSnapshot Contains deflated serialized ReplicationPushRequest for some revision. Root entity. ID is corresponding revision number (as integer). Immutable.

func GetAuthDBSnapshot 

func GetAuthDBSnapshot(ctx context.Context, rev int64, skipBody bool, dryRun bool) (*AuthDBSnapshot, error)

GetAuthDBSnapshot returns the AuthDBSnapshot datastore entity.

Returns datastore.ErrNoSuchEntity if the AuthDBSnapshot is not present. Returns an annotated error for other errors.

func (*AuthDBSnapshot) ToProto 

func (snapshot *AuthDBSnapshot) ToProto() *rpcpb.Snapshot

ToProto converts the AuthDBSnapshot entity to the protobuffer equivalent Snapshot.

type AuthDBSnapshotLatest 

type AuthDBSnapshotLatest struct {
	Kind string `gae:"$kind,AuthDBSnapshotLatest"`
	ID   string `gae:"$id"`

	// AuthDBRev is the revision number of latest stored AuthDBSnaphost. Monotonically increases.
	AuthDBRev int64 `gae:"auth_db_rev,noindex"`

	// AuthDBSha256 is the SHA256 hex digest of auth_db (before compression).
	AuthDBSha256 string `gae:"auth_db_sha256,noindex"`

	// CreatedTS is when this snapshot was created.
	ModifiedTS time.Time `gae:"modified_ts,noindex"`
}

AuthDBSnapshotLatest is a Pointer to latest stored AuthDBSnapshot entity. Exists in single instance with key ('AuthDBSnapshotLatest', 'latest').

func GetAuthDBSnapshotLatest 

func GetAuthDBSnapshotLatest(ctx context.Context, dryRun bool) (*AuthDBSnapshotLatest, error)

GetAuthDBSnapshotLatest returns the AuthDBSnapshotLatest datastore entity.

Returns datastore.ErrNoSuchEntity if the AuthDBSnapshotLatest is not present. Returns an annotated error for other errors.

type AuthGlobalConfig 

type AuthGlobalConfig struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthGlobalConfig"`
	ID   string `gae:"$id,root"`

	// OAuthClientID is an OAuth2 client_id to use to mint new OAuth2 tokens.
	OAuthClientID string `gae:"oauth_client_id,noindex"`

	// OAuthAdditionalClientIDs are the additional client ID's allowed to access
	// the service.
	OAuthAdditionalClientIDs []string `gae:"oauth_additional_client_ids,noindex"`

	// OAuthClientSecret is passed to clients.
	OAuthClientSecret string `gae:"oauth_client_secret,noindex"`

	// TokenServerURL is the URL of the token server to use to generate delegation
	// tokens.
	TokenServerURL string `gae:"token_server_url,noindex"`

	// SecurityConfig is serialized security config from security_config.proto.
	//
	// See https://source.chromium.org/chromium/infra/infra/+/main:luci/appengine/components/components/auth/proto/security_config.proto.
	SecurityConfig []byte `gae:"security_config,noindex"`
}

AuthGlobalConfig is the root entity for auth models. There should be only one instance of this model in datastore.

func GetAuthGlobalConfig 

func GetAuthGlobalConfig(ctx context.Context) (*AuthGlobalConfig, error)

GetAuthGlobalConfig returns the AuthGlobalConfig datastore entity.

Returns datastore.ErrNoSuchEntity if the AuthGlobalConfig is not present. Returns an annotated error for other errors.

type AuthGroup 

type AuthGroup struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthGroup"`
	ID   string `gae:"$id"`

	// Parent is RootKey().
	Parent *datastore.Key `gae:"$parent"`

	// Members is the list of members that are explicitly in this group.
	Members []string `gae:"members"`

	// Globs is the list of identity-glob expressions in this group.
	//
	// Example: ('user@example.com').
	Globs []string `gae:"globs"`

	// Nested is the list of nested group names.
	Nested []string `gae:"nested"`

	// Description is a human readable description of this group.
	Description string `gae:"description,noindex"`

	// Owners is the name of the group that owns this group.
	// Owners can modify or delete this group.
	Owners string `gae:"owners"`

	// CreatedTS is the time when this group was created, in UTC.
	CreatedTS time.Time `gae:"created_ts"`

	// CreatedBy is the email of the user who created this group.
	CreatedBy string `gae:"created_by"`
}

AuthGroup is a group of identities, the entity id is the group name.

func AuthGroupFromProto 

func AuthGroupFromProto(ctx context.Context, grouppb *rpcpb.AuthGroup) *AuthGroup

AuthGroupFromProto allocates a new AuthGroup using the values in the supplied protobuffer.

func CreateAuthGroup 

func CreateAuthGroup(ctx context.Context, group *AuthGroup, historicalComment string, dryRun bool) (*AuthGroup, error)

CreateAuthGroup creates a new AuthGroup and writes it to the datastore. Only the following fields will be read from the input: ID, Description, Owners, Members, Globs, Nested.

Note: creating external groups using this function is forbidden, as they are handled by the importer. See importer.go in this package.

func GetAllAuthGroups 

func GetAllAuthGroups(ctx context.Context) ([]*AuthGroup, error)

GetAllAuthGroups returns all the AuthGroups from the datastore.

Returns an annotated error.

func GetAuthGroup 

func GetAuthGroup(ctx context.Context, groupName string) (*AuthGroup, error)

GetAuthGroup gets the AuthGroup with the given id(groupName).

Returns datastore.ErrNoSuchEntity if the group given is not present. Returns an annotated error for other errors.

func UpdateAuthGroup 

func UpdateAuthGroup(ctx context.Context, groupUpdate *AuthGroup, updateMask *fieldmaskpb.FieldMask, etag, historicalComment string, dryRun bool) (*AuthGroup, error)

UpdateAuthGroup updates the given AuthGroup. The ID of the AuthGroup is used to determine which group to update. The field mask determines which fields of the AuthGroup will be updated. If the field mask is nil, all fields will be updated.

Possible errors: 

ErrInvalidArgument if the field mask provided is invalid.
ErrInvalidIdentity if any identities or globs specified in the update are invalid.
datastore.ErrNoSuchEntity if the specified group does not exist.
ErrPermissionDenied if the caller is not allowed to update the group.
ErrConcurrentModification if the provided etag is not up-to-date.
Annotated error for other errors.

func (*AuthGroup) ToProto 

func (group *AuthGroup) ToProto(ctx context.Context, includeMemberships bool) (*rpcpb.AuthGroup, error)

ToProto converts the AuthGroup entity to the protobuffer equivalent.

Set includeMemberships to true if we also want to include the group members, or false if we only want the metadata (e.g. if we're just listing groups).

type AuthIPAllowlist 

type AuthIPAllowlist struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthIPWhitelist"`
	ID   string `gae:"$id"`

	// Parent is RootKey().
	Parent *datastore.Key `gae:"$parent"`

	// Subnets is the list of subnets for this allowlist.
	Subnets []string `gae:"subnets"`

	// Description is a human readable description of this allowlist.
	Description string `gae:"description,noindex"`

	// CreatedTS is the time when this allowlist was created, in UTC.
	CreatedTS time.Time `gae:"created_ts"`

	// CreatedBy is the email of the user who created this allowlist.
	CreatedBy string `gae:"created_by"`
}

func GetAllAuthIPAllowlists 

func GetAllAuthIPAllowlists(ctx context.Context) ([]*AuthIPAllowlist, error)

GetAllAuthIPAllowlists returns all the AuthIPAllowlist entities from the datastore, these are stored as AuthIPWhitelist in the datastore.

Returns an annotated error.

func GetAuthIPAllowlist 

func GetAuthIPAllowlist(ctx context.Context, allowlistName string) (*AuthIPAllowlist, error)

GetAuthIPAllowlist gets the AuthIPAllowlist with the given allowlist name.

Returns datastore.ErrNoSuchEntity if the allowlist given is not present. Returns an annotated error for other errors.

func (*AuthIPAllowlist) ToProto 

func (allowlist *AuthIPAllowlist) ToProto() *rpcpb.Allowlist

ToProto converts the AuthIPAllowlist entity to the protobuffer equivalent Allowlist.

type AuthProjectRealms 

type AuthProjectRealms struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthProjectRealms"`
	ID   string `gae:"$id"`

	Parent *datastore.Key `gae:"$parent"`

	// Realms is all the realms of a project, see comment for AuthProjectRealms.
	// Note: to avoid hitting the Datastore limit of the maximimum size of an
	// unindexed property, the realms are compressed before being stored.
	Realms []byte `gae:"realms,noindex"`

	// ConfigRev is the git revision the config was picked up from.
	ConfigRev string `gae:"config_rev,noindex"`

	// PermsRev is the revision of permissions DB used to expand roles.
	PermsRev string `gae:"perms_rev,noindex"`
}

AuthProjectRealms is all realms of a single LUCI project.

They are defined as Realms proto message reduced to a single project:

  • Only project's realms are listed in `realms` field.
  • Only permissions used by the project are listed in `permissions` field.
  • Permissions have their metadata stripped, they have only names.

func GetAllAuthProjectRealms 

func GetAllAuthProjectRealms(ctx context.Context) ([]*AuthProjectRealms, error)

GetAllAuthProjectRealms returns all the AuthProjectRealms entities in datastore.

Returns an annotated error.

func GetAuthProjectRealms 

func GetAuthProjectRealms(ctx context.Context, project string) (*AuthProjectRealms, error)

GetAuthProjectRealms returns the AuthProjectRealms datastore entity for a given project.

Returns datastore.ErrNoSuchEntity if the AuthProjectRealms entity is not present. Returns an annotated error for other errors.

func (*AuthProjectRealms) RealmsToProto 

func (apr *AuthProjectRealms) RealmsToProto() (projectRealms *protocol.Realms, err error)

RealmsToProto processes the Realms field and returns it as a proto.

Returns an annotated error if one occurs.

type AuthProjectRealmsMeta 

type AuthProjectRealmsMeta struct {
	Kind string `gae:"$kind,AuthProjectRealmsMeta"`
	ID   string `gae:"$id,meta"`

	Parent *datastore.Key `gae:"$parent"`

	// ConfigRev is the revision the config was picked up from.
	ConfigRev string `gae:"config_rev,noindex"`

	// PermsRev is the revision of permissions.cfg used to expand roles.
	PermsRev string `gae:"perms_rev,noindex"`

	// ConfigDigest is a SHA256 digest of the raw config body.
	ConfigDigest string `gae:"config_digest,noindex"`

	// ModifiedTS is when it was last updated, (mostly FYI).
	ModifiedTS time.Time `gae:"modified_ts,noindex"`
}

AuthProjectRealmsMeta is the metadata of some AuthProjectRealms entity.

Always created/deleted/updated transactionally with the corresponding AuthProjectRealms entity, but it is not part of AuthDB itself.

Used to hold bookkeeping state related to realms.cfg processing. Can be fetched very efficiently (compared to fetching AuthProjectRealms).

ID is always meta, the parent entity is the corresponding AuthProjectRealms.

func GetAllAuthProjectRealmsMeta 

func GetAllAuthProjectRealmsMeta(ctx context.Context) ([]*AuthProjectRealmsMeta, error)

GetAllAuthProjectRealmsMeta returns all the AuthProjectRealmsMeta entities in datastore.

Returns an annotated error.

func GetAuthProjectRealmsMeta 

func GetAuthProjectRealmsMeta(ctx context.Context, project string) (*AuthProjectRealmsMeta, error)

GetAuthProjectRealmsMeta returns the AuthProjectRealmsMeta datastore entity for a given project.

Returns datastore.ErrNoSuchEntity if the AuthProjectRealmsMeta is not present. Returns an annotated error for other errors.

func (*AuthProjectRealmsMeta) ProjectID 

func (aprm *AuthProjectRealmsMeta) ProjectID() (string, error)

ProjectID returns the project that an AuthProjectRealmsMeta is connected to.

type AuthRealmsGlobals 

type AuthRealmsGlobals struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthRealmsGlobal"`
	ID   string `gae:"$id,globals"`

	Parent *datastore.Key `gae:"$parent"`

	// TODO(cjacomet): Remove this once Python version no longer depends on
	// this.
	//
	// Permissions and PermissionsList are the same, they are just represented
	// differently in datastore from Python to Golang. In Python lists of pointers
	// to protobuf types would serialize automagically and store as an array of
	// serialized strings in datastore. In Go this is not supported in the luci datastore
	// package. It is better to define the list as repeated protobuf.Msg instead of
	// []*protobuf.Msg. This will yield the same results in the end but with a bit different steps.
	// Permissions will not be unmarshalled in this version, the declaration
	// is to avoid any issues when fetching the entity from datastore.
	//
	// Permissions is all globally defined permissions, in alphabetical order.
	Permissions []string `gae:"permissions,noindex"`

	// PermissionsList is all globally defined permissions, in alphabetical order.
	PermissionsList *permissions.PermissionsList `gae:"permissionslist"`
}

AuthRealmsGlobals is a singleton entity with global portions of realms configuration.

Data here does not relate to any individual realm or project. Currently contains a list of all defined permissions (with their metadata).

func GetAuthRealmsGlobals 

func GetAuthRealmsGlobals(ctx context.Context) (*AuthRealmsGlobals, error)

GetAuthRealmsGlobals returns the AuthRealmsGlobals singleton datastore entity.

Returns datastore.ErrNoSuchEntity if the AuthRealmsGlobals entity is not present. Returns an annotated error for other errors.

type AuthReplicaState 

type AuthReplicaState struct {
	Kind string `gae:"$kind,AuthReplicaState"`
	// ID is GAE Application ID of the Replica.
	ID string `gae:"$id,self"`
	// Parent is replicasRootKey().
	Parent *datastore.Key `gae:"$parent"`

	// URL of a host to push updates to.
	ReplicaURL string `gae:"replica_url,noindex"`
	//Revision of auth DB replica is synced to.
	AuthDBRev int64 `gae:"auth_db_rev,noindex"`
	// Time when AuthDBRev was created (by primary clock).
	RevModifiedTS time.Time `gae:"rev_modified_ts,noindex"`
	// Value of components.auth.version.__version__ used by replica.
	AuthCodeVersion string `gae:"auth_code_version,noindex"`

	// Timestamp of when last push attempt started.
	PushStartedTS time.Time `gae:"push_started_ts,noindex"`
	// Timestamp of when last push attempt finished (successfully or not).
	PushFinishedTS time.Time `gae:"push_finished_ts,noindex"`
	// Status of last push attempt. See ReplicaPushStatus* enumeration.
	PushStatus ReplicaPushStatus `gae:"push_status,noindex"`
	// Error message of last push attempt, or empty string if it was successful.
	PushError string `gae:"push_error,noindex"`
}

AuthReplicaState is the last known state of a Replica as known by Auth Service.

func GetAllReplicas 

func GetAllReplicas(ctx context.Context) ([]*AuthReplicaState, error)

GetAllReplicas gets all the AuthReplicaState entities.

func GetAllStaleReplicas 

func GetAllStaleReplicas(ctx context.Context, authDBRev int64) ([]*AuthReplicaState, error)

GetAllStaleReplicas gets all the AuthReplicaState entities that are behind the given authDBRev.

func (*AuthReplicaState) ToProto 

func (r *AuthReplicaState) ToProto() *rpcpb.ReplicaState

type AuthReplicationState 

type AuthReplicationState struct {
	Kind string `gae:"$kind,AuthReplicationState"`
	ID   string `gae:"$id,self"`

	// Parent is RootKey().
	Parent *datastore.Key `gae:"$parent"`

	// AuthDBRev is increased by 1 with every change to the AuthDB that should be
	// distributed to linked services.
	AuthDBRev int64 `gae:"auth_db_rev,noindex"`

	// ModifiedTS is the time this entity was last modified.
	ModifiedTS time.Time `gae:"modified_ts,noindex"`

	PrimaryID  string   `gae:"primary_id,noindex"`
	PrimaryURL string   `gae:"primary_url,noindex"`
	ShardIDs   []string `gae:"shard_ids,noindex"`
}

AuthReplicationState contains revision number incremented each time a part of AuthDB changes.

func GetReplicationState 

func GetReplicationState(ctx context.Context) (*AuthReplicationState, error)

GetReplicationState fetches AuthReplicationState from the datastore.

Returns datastore.ErrNoSuchEntity if it is missing.

type AuthVersionedEntityMixin 

type AuthVersionedEntityMixin struct {
	// ModifiedTS is the time this entity was last modified.
	ModifiedTS time.Time `gae:"modified_ts"`

	// ModifiedBy specifies who mode the modification to this entity.
	ModifiedBy string `gae:"modified_by"`

	// AuthDBRev is the revision number that this entity was last modified at.
	AuthDBRev int64 `gae:"auth_db_rev"`

	// AuthDBPrevRev is revision number of the previous version of this entity.
	AuthDBPrevRev int64 `gae:"auth_db_prev_rev"`
}

AuthVersionedEntityMixin is for AuthDB entities that keep track of when they change.

type ChangeType 

type ChangeType int64

ChangeType is the enum for AuthDBChange.ChangeType. 

const (
	// AuthDBGroupChange change types.
	ChangeGroupCreated            ChangeType = 1000
	ChangeGroupDescriptionChanged ChangeType = 1100
	ChangeGroupOwnersChanged      ChangeType = 1150
	ChangeGroupMembersAdded       ChangeType = 1200
	ChangeGroupMembersRemoved     ChangeType = 1300
	ChangeGroupGlobsAdded         ChangeType = 1400
	ChangeGroupGlobsRemoved       ChangeType = 1500
	ChangeGroupNestedAdded        ChangeType = 1600
	ChangeGroupNestedRemoved      ChangeType = 1700
	ChangeGroupDeleted            ChangeType = 1800

	// AuthDBIPAllowlistChange change types.
	ChangeIPALCreated            ChangeType = 3000
	ChangeIPALDescriptionChanged ChangeType = 3100
	ChangeIPALSubnetsAdded       ChangeType = 3200
	ChangeIPALSubnetsRemoved     ChangeType = 3300
	ChangeIPALDeleted            ChangeType = 3400

	// AuthDBIPAllowlistAssignmentChange change types.
	ChangeIPALAssignSet   ChangeType = 5000
	ChangeIPALAssignUnset ChangeType = 5100

	// AuthDBConfigChange change types.
	ChangeConfOauthClientChanged    ChangeType = 7000
	ChangeConfClientIDsAdded        ChangeType = 7100
	ChangeConfClientIDsRemoved      ChangeType = 7200
	ChangeConfTokenServerURLChanged ChangeType = 7300
	ChangeConfSecurityConfigChanged ChangeType = 7400

	// AuthRealmsGlobalsChange change types.
	ChangeRealmsGlobalsChanged ChangeType = 9000

	// AuthProjectRealmsChange change types.
	ChangeProjectRealmsCreated     ChangeType = 10000
	ChangeProjectRealmsChanged     ChangeType = 10100
	ChangeProjectRealmsReevaluated ChangeType = 10200
	ChangeProjectRealmsRemoved     ChangeType = 10300
)

func (ChangeType) ToString 

func (ct ChangeType) ToString() string

type ExpandedRealms 

type ExpandedRealms struct {
	CfgRev *RealmsCfgRev
	// Realms is the expanded form of the realms.cfg for a project
	Realms *protocol.Realms
}

ExpandedRealms contains the expanded realms and information about the expanded realms.

type GroupBundle 

type GroupBundle = map[string][]identity.Identity

GroupBundle is a map where k: groupName, v: list of identities belonging to group k.

type GroupImporterConfig 

type GroupImporterConfig struct {
	Kind string `gae:"$kind,GroupImporterConfig"`
	ID   string `gae:"$id,config"`

	// ConfigProto is the plaintext copy of the config found at imports.cfg.
	ConfigProto string `gae:"config_proto"`

	// ConfigRevision is revision version of the config found at imports.cfg.
	ConfigRevision []byte `gae:"config_revision"`

	// ModifiedBy is the email of the user who modified the cfg.
	ModifiedBy string `gae:"modified_by,noindex"`

	// ModifiedTS is the time when this entity was last modified.
	ModifiedTS time.Time `gae:"modified_ts,noindex"`
}

GroupImporterConfig is a singleton entity that contains the contents of the imports.cfg file.

func GetGroupImporterConfig 

func GetGroupImporterConfig(ctx context.Context) (*GroupImporterConfig, error)

GetGroupImporterConfig fetches the GroupImporterConfig entity from the datastore. 

Returns GroupImporterConfig entity if present.
Returns datastore.ErrNoSuchEntity if the entity is not present.
Returns annotated error for all other errors.

func (*GroupImporterConfig) ToProto 

func (g *GroupImporterConfig) ToProto() (*configspb.GroupImporterConfig, error)

ToProto converts the GroupImporterConfig entity to the proto equivalent.

type ImportedConfigRevisions 

type ImportedConfigRevisions struct {
	Kind string `gae:"$kind,_ImportedConfigRevisions"`
	ID   string `gae:"$id,self"`

	// Parent is RootKey().
	Parent *datastore.Key `gae:"$parent"`

	// Serialized mapping of config path -> {'rev': SHA1, 'url': URL}
	Revisions []byte `gae:"revisions"`
}

ImportedConfigRevisions is the Datastore entity used by Auth Service v1 to keep track of config revision info. It is not necessary in v2, but its data should be updated so the the UI for v1 remains accurate.

type RealmsCfgRev 

type RealmsCfgRev struct {
	ProjectID    string
	ConfigRev    string
	ConfigDigest string

	// These two are mutually exclusive.
	ConfigBody []byte
	PermsRev   string
}

RealmsCfgRev is information about fetched or previously processed realms.cfg. Comes either from LUCI Config (then `ConfigBody` is set, but `PermsRev` isn't) or from the datastore (then `PermsRev` is set, but `ConfigBody` isn't). All other fields are always set.

type ReplicaPushStatus 

type ReplicaPushStatus int

ReplicaPushStatus is an enum for the replica push attempt status. 

const (
	ReplicaPushStatusSuccess ReplicaPushStatus = iota
	ReplicaPushStatusTransientError
	ReplicaPushStatusFatalError
)

The possible values for the PushStatus field of AuthReplicaState.

type ReplicaUpdateError 

type ReplicaUpdateError struct {
	RootErr error
	IsFatal bool
}

ReplicaUpdateError wraps an error that occurred when attempting to directly push a new AuthDB revision to a replica.

func (*ReplicaUpdateError) Error 

func (r *ReplicaUpdateError) Error() string

func (*ReplicaUpdateError) Is 

func (r *ReplicaUpdateError) Is(target error) bool

func (*ReplicaUpdateError) Unwrap 

func (r *ReplicaUpdateError) Unwrap() error

type Snapshot 

type Snapshot struct {
	ReplicationState *AuthReplicationState
	GlobalConfig     *AuthGlobalConfig
	Groups           []*AuthGroup
	IPAllowlists     []*AuthIPAllowlist
	RealmsGlobals    *AuthRealmsGlobals
	ProjectRealms    []*AuthProjectRealms
}

Snapshot contains transactionally captured AuthDB entities.

func TakeSnapshot 

func TakeSnapshot(ctx context.Context) (snap *Snapshot, err error)

TakeSnapshot takes a consistent snapshot of the replicated subset of AuthDB entities.

Runs a read-only transaction internally.

func (*Snapshot) ToAuthDB 

func (s *Snapshot) ToAuthDB(ctx context.Context, useV1Perms bool) (*authdb.SnapshotDB, error)

ToAuthDB converts the snapshot to an authdb.SnapshotDB.

It then can be used by the auth service itself to make ACL checks.

func (*Snapshot) ToAuthDBProto 

func (s *Snapshot) ToAuthDBProto(ctx context.Context, useV1Perms bool) (*protocol.AuthDB, error)

ToAuthDBProto converts the snapshot to an AuthDB proto message.

Source Files

View all Source files

Directories

Path Synopsis
Package graph contains groups graph definitions and operations.
 Package graph contains groups graph definitions and operations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.