worker

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 13, 2024 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Overview

Package worker provides worker functions and types that allow callers to abstract away the lower-level details of the various operations needed for the Managed Tokens utilities. Ideally, most callers should just need to set up worker.Config objects using worker.NewConfig, obtain the worker channels using worker.NewChannelsForWorkers, and call the applicable worker with the above ChannelsForWorkers object. All that remains then for the caller is to pass the worker.Config objects into the ChannelsForWorkers.GetServiceConfigChan(), and listen on the ChannelsForWorkers.GetSuccessChan() and ChannelsForWorkers.GetNotificationsChan()

package worker contains the various workers that can be used concurrently to perform the various operations supported by the managed tokens library

Index

Constants

View Source 
const (
	// DefaultRoleFileTemplate is a key to store the value of the default role file template in the Config.Extras map
	DefaultRoleFileDestinationTemplate supportedExtrasKey = iota
	FileCopierOptions
	VaultTokenStoreHoldoff
	PingOptions
	SSHOptions
)

Variables

This section is empty.

Functions

func GetDefaultRoleFileDestinationTemplateValueFromExtras 

func GetDefaultRoleFileDestinationTemplateValueFromExtras(c *Config) (string, bool)

GetDefaultRoleFileTemplateValueFromExtras retrieves the default role file template value from the worker.Config, and asserts that it is a string. Callers should check the bool return value to make sure the type assertion passes, for example: 

c := worker.NewConfig( // various options )
// set the default role file template in here
tmplString, ok := GetDefaultRoleFileTemplateValueFromExtras(c)
if !ok { // handle missing or incorrect value }

func GetFileCopierOptionsFromExtras 

func GetFileCopierOptionsFromExtras(c *Config) ([]string, bool)

GetFileCopierOptionsFromExtras retrieves the file copier options value from the worker.Config, and asserts that it is a string. Callers should check the bool return value to make sure the type assertion passes, for example: 

c := worker.NewConfig( // various options )
// set the default role file template in here
opts, ok := GetFileCopierOptionsFromExtras(c)
if !ok { // handle missing or incorrect value }

func GetKerberosTicketandVerify 

func GetKerberosTicketandVerify(ctx context.Context, sc *Config) error

func GetKerberosTicketsWorker 

func GetKerberosTicketsWorker(ctx context.Context, chans channelGroup)

GetKerberosTicketsWorker is a worker that listens on chans.GetServiceConfigChan(), and for the received worker.Config objects, obtains kerberos tickets from the configured kerberos principals. It returns when chans.GetServiceConfigChan() is closed, and it will in turn close the other chans in the passed in ChannelsForWorkers

func GetPingOptionsFromExtras 

func GetPingOptionsFromExtras(c *Config) ([]string, bool)

GetPingOptionsFromExtras retrieves the ping options slice from the worker.Config, and asserts that it is a []string. Callers should check the bool return value to make sure that the type assertion passes.

func GetSSHOptionsFromExtras 

func GetSSHOptionsFromExtras(c *Config) ([]string, bool)

GetSSHOptionsFromExtras retrieves the SSH options slice from the worker.Config, and asserts that it is a []string. Callers should check the bool return value to make sure that the type assertion passes.

func GetVaultTokenStoreHoldoff 

func GetVaultTokenStoreHoldoff(c *Config) (holdoff bool, ok bool)

GetVaultTokenStoreHoldoff returns the value from the Config for the Extras VaultTokenStoreHoldoff key. It also returns a bool, ok, indicating whether this value should be used or not.

func NewChannelsForWorkers 

func NewChannelsForWorkers(bufferSize int) channelGroup

NewChannelsForWorkers returns a ChannelsForWorkers that is initialized and ready to pass to workers and listen on

func PingAggregatorWorker 

func PingAggregatorWorker(ctx context.Context, chans channelGroup)

PingAggregatorWorker is a worker that listens on chans.GetServiceConfigChan(), and for the received worker.Config objects, concurrently pings all of the Config's destination nodes. It returns when chans.GetServiceConfigChan() is closed, and it will in turn close the other chans in the passed in ChannelsForWorkers

func PushTokensWorker 

func PushTokensWorker(ctx context.Context, chans channelGroup)

PushTokenWorker is a worker that listens on chans.GetServiceConfigChan(), and for the received worker.Config objects, pushes vault tokens to all the configured destination nodes. It returns when chans.GetServiceConfigChan() is closed, and it will in turn close the other chans in the passed in ChannelsForWorkers

func SetSupportedExtrasKeyValue 

func SetSupportedExtrasKeyValue(key supportedExtrasKey, value any) func(*Config) error

SetSupportedExtrasKeyValue returns a func(*Config) that sets the value for the given supportedExtraskey in the Extras map

func SetVaultTokenStoreHoldoff 

func SetVaultTokenStoreHoldoff() func(*Config) error

SetVaultTokenStoreHoldoff returns a func(*Config) that sets the VaultTokenStoreHoldoff Extras key of the *Config to true

func StoreAndGetTokenInteractiveWorker added in v0.16.0 

func StoreAndGetTokenInteractiveWorker(ctx context.Context, chans channelGroup)

StoreAndGetTokenInteractiveWorker is a worker that listens on chans.GetServiceConfigChan(), and for the received worker.Config objects, stores a refresh token in the configured vault and obtains vault and bearer tokens. It is meant to be used for a single interactive operation to store a single service's vault token on the configured credds. It will display all the stdout from the underlying executables to screen.

func StoreAndGetTokenWorker 

func StoreAndGetTokenWorker(ctx context.Context, chans channelGroup)

StoreAndGetTokenWorker is a worker that listens on chans.GetServiceConfigChan(), and for the received worker.Config objects, stores a refresh token in the configured vault and obtains vault and bearer tokens. It returns when chans.GetServiceConfigChan() is closed, and it will in turn close the other chans in the passed in ChannelsForWorkers

func StoreAndGetTokensForSchedd added in v0.16.0 

func StoreAndGetTokensForSchedd[T tokenStorer](ctx context.Context, environ *environment.CommandEnvironment, tokenRootPath string, ts T) error

StoreAndGetTokensForSchedds will store a refresh token on the condor-configured vault server, obtain vault and bearer tokens for a service using HTCondor executables, and store the vault token in the condor_credd that resides on each schedd that is passed in with the schedds slice. If there was an error with ANY of the schedds, StoreAndGetTokensForSchedds will return an error

Types

type Config 

type Config struct {
	service.Service
	UserPrincipal string   // The principal of the kerberos credential needed by the various workers
	Nodes         []string // The destination nodes for PushTokensWorker to push copies of the vault tokens to
	Account       string   // The user account the PushTokensWorker should use to connect to a destination node
	KeytabPath    string   // The path on disk where the kerberos keytab is stored
	// The directory on disk where the vault tokens per credd are stored.  If this path is given, and either does not
	// exist or does not have the relevant token, the utility will assume that the file/directory does not exist and
	// create it
	ServiceCreddVaultTokenPathRoot string
	DesiredUID                     uint32   // The UID associated with the Account. This determines the destination filename
	Schedds                        []string // The list of schedds/credds where a StoreAndGetTokenWorker should store vault tokens
	VaultServer                    string   // The vault server hosting the Hashicorp Vault that the refresh token should be saved to
	// Extras is a map where any value can be stored that may not fit into the above categories.
	// To allow an external package to set an Extras value, define an exported func that sets
	// the value directly.  For example:
	//  func SetSupportedExtrasKeyValue(c *Config, key supportedExtrasKey, value any) { c.Extras[key] = value }
	//
	// The functional option version of this would look like:
	//  func SetSupportedExtrasKeyValue(key supportedExtrasKey, value any) func (*Config) { return func(c *Config) {c.Extras[key] = value } }
	//
	// It's also a good idea to create a getter value for each supportedExtrasKey, so that type checks can be done.  For example, for a key whose
	// value's underlying type we expect to be a string, define a func like this:
	//  func GetMySupportedExtrasKey(c *Config) (string, bool) {
	// 		if val, ok := c.Extras[MySupportedKey].(string); ok {
	//  			return val, ok
	//  	}
	//  }
	// Then the caller should check ok to make sure it's true before using the value
	Extras map[supportedExtrasKey]any

	environment.CommandEnvironment
	// contains filtered or unexported fields
}

Config is a mega struct containing all the information the workers need to have or pass onto lower level funcs.

func NewConfig 

func NewConfig(service service.Service, options ...ConfigOption) (*Config, error)

NewConfig takes the config information from the global file and creates an *Config object To create functional options, simply define functions that operate on an *Config. E.g. func foo(e *Config) { e.Name = "bar" }. You can then pass in foo to CreateConfig (e.g. NewConfig("my_expt", foo), to set the Config.Name to "bar".

To pass in something that's dynamic, define a function that returns a func(*Config). e.g.: 

func foo(bar int, e *Config) func(*Config) {
	baz = bar + 3
	return func(*Config) {
		e.spam = baz
	}

If you then pass in foo(3), like NewConfig("my_expt", foo(3)), then Config.spam will be set to 6 Borrowed heavily from https://cdcvs.fnal.gov/redmine/projects/discompsupp/repository/ken_proxy_push/revisions/master/entry/utils/experimentConfig.go

func (*Config) IsNodeUnpingable 

func (c *Config) IsNodeUnpingable(node string) bool

IsNodeUnpingable checks the Config's unPingableNodes field to see if a node is registered there

func (*Config) RegisterUnpingableNode 

func (c *Config) RegisterUnpingableNode(node string)

RegisterUnpingableNode registers a node in the Config's unPingableNodes field

func (*Config) ServiceNameFromExperimentAndRole 

func (c *Config) ServiceNameFromExperimentAndRole() string

ServiceNameFromExperimentAndRole returns a reconstructed service name by concatenating the underlying Service.Experiment() value, "_", and the underlying Service.Role() value. This is useful in case the caller has overridden the experiment or role name in the case of duplicate services that have different configurations (e.g. the same vault token needs to be pushed to two sets of credds in two different pools) In general, this should be used in lieu of Config.Service.Name() for notifications passing

type ConfigOption 

type ConfigOption func(*Config) error

ConfigOption is a functional option that should be used as an argument to NewConfig to set various fields of the Config For example: 

 f := func(c *Config) error {
	  c.Account = "myaccount"
   return nil
 }
 g := func(c *Config) error {
	  c.DesiredUID = 42
   return nil
 }
 config := NewConfig("myservice", f, g)

func SetAccount 

func SetAccount(value string) ConfigOption

func SetCommandEnvironment 

func SetCommandEnvironment(cmdEnvFuncs ...func(e *environment.CommandEnvironment)) ConfigOption

SetCommandEnvironment is a helper func that takes a variadic of func(*environment.CommandEnvironment) and returns a func(*Config) that sets the Config's embedded CommandEnvironment by running the funcs passed in the variadic. It is meant to be used to as a functional opt in the call to NewConfig to create a new Config object. For example: 

c := NewConfig(
  SetCommandEnvironment(
    func(e *environment.CommandEnvironment) { e.SetCondorCreddHost("my_credd_host") }
  )
)

func SetDesiredUID 

func SetDesiredUID(value uint32) ConfigOption

func SetKeytabPath 

func SetKeytabPath(value string) ConfigOption

func SetNodes 

func SetNodes(value []string) ConfigOption

func SetSchedds 

func SetSchedds(value []string) ConfigOption

func SetServiceCreddVaultTokenPathRoot 

func SetServiceCreddVaultTokenPathRoot(value string) ConfigOption

func SetUserPrincipal 

func SetUserPrincipal(value string) ConfigOption

func SetVaultServer 

func SetVaultServer(value string) ConfigOption

func SetWorkerRetryValue added in v0.15.0 

func SetWorkerRetryValue(w WorkerType, value uint) ConfigOption

SetWorkerRetryValue is a function that sets the retry value for a specific worker type. It returns a ConfigOption

type SuccessReporter 

type SuccessReporter interface {
	GetService() service.Service
	GetSuccess() bool
}

SuccessReporter is an interface to objects that report success or failures from various workers

type UIDEntryFromFerry 

type UIDEntryFromFerry struct {
	// contains filtered or unexported fields
}

UIDEntryFromFerry is an entry that represents data returned from the FERRY getUserInfo API. It implements utils.FerryUIDDatum

func GetFERRYUIDData 

func GetFERRYUIDData(ctx context.Context, username string, ferryHost string, ferryPort int,
	requestRunnerWithAuthMethodFunc func(ctx context.Context, url, verb string) (*http.Response, error),
	ferryDataChan chan<- db.FerryUIDDatum) (*UIDEntryFromFerry, error)

GetFERRYUIDData queries FERRY for user information. This func abstracts away the actual details of formulating the HTTP request, including authentication, headers, etc. All of these details must be provided in the requestRunnerWithAuthMethodFunc func that is passed in. An example from a caller could look like this:

// myauthfunc sends a request to a server without any authentication func myauthfunc(ctx context.Context, url, verb string) (*http.Response, error){ 

		client := &http.Client{}
		req, err := http.NewRequest(verb, url, nil)
		if err != nil {}
		resp, err := client.Do(req)
		return resp, err
}

ctx := context.Background() myentry, err := GetFERRYUIDData(ctx, "user1", "https://example.com", 0, myauthfunc)

func (*UIDEntryFromFerry) String 

func (u *UIDEntryFromFerry) String() string

func (*UIDEntryFromFerry) Uid 

func (u *UIDEntryFromFerry) Uid() int

func (*UIDEntryFromFerry) Username 

func (u *UIDEntryFromFerry) Username() string

type Worker added in v0.16.0 

type Worker func(context.Context, channelGroup)

Worker is a function, controlled by a context, that performs some operations based on the values passed through the channelGroup.GetServiceConfigChan() channel. The worker should report success or failure through the channelGroup.GetSuccessChan()

type WorkerType added in v0.15.0 

type WorkerType uint8

WorkerType is a type that represents the kind of worker being referenced. Its main use is to set configuration values that are worker-specific, like retry counts, timeouts, etc. 

const (
	GetKerberosTicketsWorkerType WorkerType = iota
	StoreAndGetTokenWorkerType
	StoreAndGetTokenInteractiveWorkerType
	PingAggregatorWorkerType
	PushTokensWorkerType
)

func (WorkerType) String added in v0.15.0 

func (wt WorkerType) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.