steampipeconfig

package
v0.20.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 22, 2023 License: AGPL-3.0 Imports: 43 Imported by: 1

Documentation

Index

Constants

View Source 
const PostgresNotificationStructVersion = 20230306

Variables

View Source 
var ConnectionStateTableAddedColumns map[string]string = map[string]string{
	"connections": "TEXT[]",
}

ConnectionStateTableAddedColumns is a map of column names to the SQL type these are columns which needed to be added after 0.20.0 with the steampipe_connection_state table was released

these columns are loaded optionally when loading up the connection state table make sure these columns are NULLABLE

View Source 
var GlobalWorkspaceProfile *modconfig.WorkspaceProfile
View Source 
var WithWaitForSearchPath = func(searchPath []string) func(config *LoadConnectionStateConfiguration) {
	return func(config *LoadConnectionStateConfiguration) {
		config.WaitMode = WaitForSearchPath
		config.SearchPath = searchPath
	}
}
View Source 
var WithWaitUntilLoading = func() func(config *LoadConnectionStateConfiguration) {
	return func(config *LoadConnectionStateConfiguration) {
		config.WaitMode = WaitForLoading
	}
}

WithWaitUntilLoading waits until no connections are in pending state

View Source 
var WithWaitUntilReady = func(connections ...string) func(config *LoadConnectionStateConfiguration) {
	return func(config *LoadConnectionStateConfiguration) {
		config.Connections = connections
		config.WaitMode = WaitForReady
	}
}

WithWaitUntilReady waits until all are in ready state

Functions

func BuildValidationWarningString 

func BuildValidationWarningString(failures []*ValidationFailure) string

func DeleteConnectionStateFile added in v0.20.0 

func DeleteConnectionStateFile()

func GetLoadingConnectionStatusMessage added in v0.20.0 

func GetLoadingConnectionStatusMessage(connectionStateMap ConnectionStateMap, requiredSchemas ...string) string

func GetModFileExtensions 

func GetModFileExtensions() []string

GetModFileExtensions returns list of all file extensions we care about this will be the mod data extension, plus any registered extensions registered in fileToResourceMap

func GetVariableValues 

func GetVariableValues(ctx context.Context, parseCtx *parse.ModParseContext, variableMap *modconfig.ModVariableMap, validate bool) (*modconfig.ModVariableMap, *modconfig.ErrorAndWarnings)

func IsCloudWorkspaceIdentifier added in v0.17.0 

func IsCloudWorkspaceIdentifier(name string) bool

IsCloudWorkspaceIdentifier returns whether name is a cloud workspace identifier of the form: {identity_handle}/{workspace_handle},

func LoadMod 

func LoadMod(modPath string, parseCtx *parse.ModParseContext) (mod *modconfig.Mod, errorsAndWarnings *modconfig.ErrorAndWarnings)

LoadMod parses all hcl files in modPath and returns a single mod if CreatePseudoResources flag is set, construct hcl resources for files with specific extensions NOTE: it is an error if there is more than 1 mod defined, however zero mods is acceptable - a default mod will be created assuming there are any resource files

func LoadModResourceNames 

func LoadModResourceNames(mod *modconfig.Mod, parseCtx *parse.ModParseContext) (resources *modconfig.WorkspaceResources, err error)

LoadModResourceNames parses all hcl files in modPath and returns the names of all resources

func LoadVariableDefinitions 

func LoadVariableDefinitions(variablePath string, parseCtx *parse.ModParseContext) (*modconfig.ModVariableMap, error)

func NewConnectionUpdates 

func NewConnectionUpdates(ctx context.Context, pool *pgxpool.Pool, forceUpdateConnectionNames ...string) (*ConnectionUpdates, *RefreshConnectionResult)

NewConnectionUpdates returns updates to be made to the database to sync with connection config

func SaveConnectionStateFile added in v0.20.0 

func SaveConnectionStateFile(res *RefreshConnectionResult, connectionUpdates *ConnectionUpdates)

func ValidateConnectionName added in v0.20.0 

func ValidateConnectionName(connectionName string) error

Types

type ActorMetadata 

type ActorMetadata struct {
	Id     string `json:"id,omitempty"`
	Handle string `json:"handle,omitempty"`
}

type CloudMetadata 

type CloudMetadata struct {
	Actor             *ActorMetadata     `json:"actor,omitempty"`
	Identity          *IdentityMetadata  `json:"identity,omitempty"`
	WorkspaceDatabase *WorkspaceMetadata `json:"workspace,omitempty"`
	ConnectionString  string             `json:"-"`
}

type ConnectionPlugin 

type ConnectionPlugin struct {
	// map of connection data (name, config, options)
	// keyed by connection name
	ConnectionMap       map[string]*ConnectionPluginData
	PluginName          string
	PluginClient        *sdkgrpc.PluginClient
	SupportedOperations *proto.SupportedOperations
}

ConnectionPlugin is a structure representing an instance of a plugin for non-legacy plugins, each plugin instance supportds multiple connections the config, options and schema for each connection is stored in ConnectionMap

func NewConnectionPlugin added in v0.16.0 

func NewConnectionPlugin(pluginName string, pluginClient *sdkgrpc.PluginClient, supportedOperations *proto.SupportedOperations) *ConnectionPlugin

func (ConnectionPlugin) GetSchema added in v0.19.0 

func (p ConnectionPlugin) GetSchema(connectionName string) (*sdkproto.Schema, error)

GetSchema returns the cached schema if it is static, or if it is dynamic, refetch it

func (ConnectionPlugin) IncludesConnection added in v0.16.0 

func (p ConnectionPlugin) IncludesConnection(name string) bool

type ConnectionPluginData added in v0.16.0 

type ConnectionPluginData struct {
	Name    string
	Config  string
	Type    string
	Options *options.Connection
	Schema  *sdkproto.Schema
}

type ConnectionSchemaMap 

type ConnectionSchemaMap map[string][]string

ConnectionSchemaMap is a map of connection to all connections with the same schema key is exemplar connection and value is all connections with same schema

func NewConnectionSchemaMap 

func NewConnectionSchemaMap(ctx context.Context, connectionStateMap ConnectionStateMap, searchPath []string) ConnectionSchemaMap

NewConnectionSchemaMap creates a ConnectionSchemaMap for all configured connections this is a map keyed by exemplar connection with the value the connections which have the same schema it uses the current connection state to determine if a connection has a dynamic schema

type ConnectionState added in v0.20.0 

type ConnectionState struct {
	// the connection name
	ConnectionName string `json:"connection,omitempty"  db:"name"`
	// connection type (expected value: "aggregator")
	Type *string `json:"type,omitempty"  db:"type"`
	// should we create apostgres schema for the connection (expected values: "enable", "disable")
	ImportSchema string `json:"import_schema"  db:"import_schema"`
	// the fully qualified name of the plugin
	Plugin string `json:"plugin,omitempty"  db:"plugin"`
	// the connection state (pending, updating, deleting, error, ready)
	State string `json:"state,omitempty"  db:"state"`
	// error (if there is one - make a pointer to support null)
	ConnectionError *string `json:"error,omitempty" db:"error"`
	// schema mode - static or dynamic
	SchemaMode string `json:"schema_mode,omitempty" db:"schema_mode"`
	// the hash of the connection schema - this is used to determine if a dynamic schema has changed
	SchemaHash string `json:"schema_hash,omitempty" db:"schema_hash"`
	// are the comments set
	CommentsSet bool `json:"comments_set" db:"comments_set"`
	// the creation time of the plugin file
	PluginModTime time.Time `json:"plugin_mod_time" db:"plugin_mod_time"`
	// the update time of the connection
	ConnectionModTime time.Time `json:"connection_mod_time" db:"connection_mod_time"`
	// the matching patterns of child connections (for aggregators)
	Connections []string `json:"connections" db:"connections"`
}

ConnectionState is a struct containing all details for a connection - the plugin name and checksum, the connection config and options json tags needed as this is stored in the connection state file

func NewConnectionState added in v0.20.0 

func NewConnectionState(remoteSchema string, connection *modconfig.Connection, creationTime time.Time) *ConnectionState

func (*ConnectionState) CanCloneSchema added in v0.20.0 

func (d *ConnectionState) CanCloneSchema() bool

func (*ConnectionState) Disabled added in v0.20.0 

func (d *ConnectionState) Disabled() bool

func (*ConnectionState) Equals added in v0.20.0 

func (d *ConnectionState) Equals(other *ConnectionState) bool

func (*ConnectionState) Error added in v0.20.0 

func (d *ConnectionState) Error() string

func (*ConnectionState) GetType added in v0.20.0 

func (d *ConnectionState) GetType() string

func (*ConnectionState) Loaded added in v0.20.0 

func (d *ConnectionState) Loaded() bool

Loaded returns true if the connection state is 'ready' or 'error' Disabled connections are considered as 'loaded'

func (*ConnectionState) SetError added in v0.20.0 

func (d *ConnectionState) SetError(err string)

type ConnectionStateMap added in v0.20.0 

type ConnectionStateMap map[string]*ConnectionState

func GetRequiredConnectionStateMap added in v0.20.0 

func GetRequiredConnectionStateMap(connectionMap map[string]*modconfig.Connection, currentConnectionState ConnectionStateMap) (ConnectionStateMap, map[string][]modconfig.Connection, error)

GetRequiredConnectionStateMap populates a map of connection data for all connections in connectionMap

func LoadConnectionState added in v0.20.0 

func LoadConnectionState(ctx context.Context, conn *pgx.Conn, opts ...LoadConnectionStateOption) (ConnectionStateMap, error)

LoadConnectionState populates a ConnectionStateMap from the connection_state table it verifies the table has been initialised by calling RefreshConnections after db startup

func (ConnectionStateMap) ConnectionModTime added in v0.20.0 

func (m ConnectionStateMap) ConnectionModTime() time.Time

ConnectionModTime returns the latest connection mod time

func (ConnectionStateMap) ConnectionsInState added in v0.20.0 

func (m ConnectionStateMap) ConnectionsInState(states ...string) bool

ConnectionsInState returns whether there are any connections one of the given states

func (ConnectionStateMap) Equals added in v0.20.0 

func (m ConnectionStateMap) Equals(other ConnectionStateMap) bool

func (ConnectionStateMap) GetFirstSearchPathConnectionForPlugins added in v0.20.0 

func (m ConnectionStateMap) GetFirstSearchPathConnectionForPlugins(searchPath []string) []string

func (ConnectionStateMap) GetPluginToConnectionMap added in v0.20.0 

func (m ConnectionStateMap) GetPluginToConnectionMap() map[string][]string

func (ConnectionStateMap) GetSummary added in v0.20.0 

func (m ConnectionStateMap) GetSummary() ConnectionStateSummary

func (ConnectionStateMap) Loaded added in v0.20.0 

func (m ConnectionStateMap) Loaded(connections ...string) bool

Loaded returns whether loading is complete, i.e. all connections are either ready or error (optionally, a list of connections may be passed, in which case just these connections are checked)

func (ConnectionStateMap) Pending added in v0.20.0 

func (m ConnectionStateMap) Pending() bool

Pending returns whether there are any connections in the map which are pending this indicates that the db has just started and RefreshConnections has not been called yet

func (ConnectionStateMap) Save added in v0.20.0 

func (m ConnectionStateMap) Save() error

type ConnectionStateSummary added in v0.20.0 

type ConnectionStateSummary map[string]int

type ConnectionUpdates 

type ConnectionUpdates struct {
	Update          ConnectionStateMap
	Delete          map[string]struct{}
	Disabled        map[string]struct{}
	MissingComments ConnectionStateMap

	MissingPlugins map[string][]modconfig.Connection
	// the connections which will exist after the update
	FinalConnectionState ConnectionStateMap
	// connection plugins required to perform the updates, keyed by connection name
	ConnectionPlugins      map[string]*ConnectionPlugin
	CurrentConnectionState ConnectionStateMap
	InvalidConnections     map[string]*ValidationFailure
}

func (*ConnectionUpdates) DynamicUpdates added in v0.20.6 

func (u *ConnectionUpdates) DynamicUpdates() []string

DynamicUpdates returns the names of all dynamic plugins which are being updated

func (*ConnectionUpdates) HasUpdates 

func (u *ConnectionUpdates) HasUpdates() bool

func (*ConnectionUpdates) IdentifyMissingComments added in v0.20.0 

func (u *ConnectionUpdates) IdentifyMissingComments()

IdentifyMissingComments identifies any connections which are not being updated/deleted but which have not got comments set NOTE: this mutates FinalConnectionState to set comment_set (if needed)

func (*ConnectionUpdates) String added in v0.17.0 

func (u *ConnectionUpdates) String() string

type DependencyPathKey added in v0.20.6 

type DependencyPathKey string

DependencyPathKey is a string representation of a dependency path

  • a set of mod dependencyPath values separated by '->'

e.g. local -> github.com/kaidaguerre/steampipe-mod-m1@v3.1.1 -> github.com/kaidaguerre/steampipe-mod-m2@v5.1.1

func (DependencyPathKey) GetParent added in v0.20.6 

func (k DependencyPathKey) GetParent() DependencyPathKey

func (DependencyPathKey) PathLength added in v0.20.6 

func (k DependencyPathKey) PathLength() int

how long is the depdency path

type IdentityMetadata 

type IdentityMetadata struct {
	Id     string `json:"id,omitempty"`
	Handle string `json:"handle,omitempty"`
	Type   string `json:"type,omitempty"`
}

type LoadConnectionStateConfiguration added in v0.20.0 

type LoadConnectionStateConfiguration struct {
	WaitMode    WaitModeValue
	Connections []string
	SearchPath  []string
}

type LoadConnectionStateOption added in v0.20.0 

type LoadConnectionStateOption = func(config *LoadConnectionStateConfiguration)

type MissingVariableError added in v0.20.6 

type MissingVariableError struct {
	MissingVariables           []*modconfig.Variable
	MissingTransitiveVariables map[DependencyPathKey][]*modconfig.Variable
	// contains filtered or unexported fields
}

func NewMissingVarsError added in v0.20.6 

func NewMissingVarsError(workspaceMod *modconfig.Mod) MissingVariableError

func (MissingVariableError) Error added in v0.20.6 

func (m MissingVariableError) Error() string

type PostgresNotification added in v0.19.0 

type PostgresNotification struct {
	StructVersion int
	Type          PostgresNotificationType
}

type PostgresNotificationType added in v0.19.0 

type PostgresNotificationType int
const (
	PgNotificationSchemaUpdate PostgresNotificationType = iota + 1
)

type RefreshConnectionResult 

type RefreshConnectionResult struct {
	modconfig.ErrorAndWarnings
	UpdatedConnections bool
	Updates            *ConnectionUpdates
	FailedConnections  map[string]string
}

RefreshConnectionResult is a structure used to contain the result of either a RefreshConnections or a NewLocalClient operation

func CreateConnectionPlugins 

func CreateConnectionPlugins(connectionNamesToCreate []string) (requestedConnectionPluginMap map[string]*ConnectionPlugin, res *RefreshConnectionResult)

CreateConnectionPlugins instantiates plugins for specified connections, and fetches schemas

func NewErrorRefreshConnectionResult added in v0.20.0 

func NewErrorRefreshConnectionResult(err error) *RefreshConnectionResult

func (*RefreshConnectionResult) AddFailedConnection added in v0.19.0 

func (r *RefreshConnectionResult) AddFailedConnection(c string, failure string)

func (*RefreshConnectionResult) Merge 

func (r *RefreshConnectionResult) Merge(other *RefreshConnectionResult)

func (*RefreshConnectionResult) String added in v0.17.0 

func (r *RefreshConnectionResult) String() string

type SteampipeConfig 

type SteampipeConfig struct {
	// map of connection name to partially parsed connection config
	Connections map[string]*modconfig.Connection

	// Steampipe options
	DefaultConnectionOptions *options.Connection
	DatabaseOptions          *options.Database
	TerminalOptions          *options.Terminal
	GeneralOptions           *options.General
	// contains filtered or unexported fields
}

SteampipeConfig is a struct to hold Connection map and Steampipe options 

var GlobalConfig *SteampipeConfig

func LoadConnectionConfig 

func LoadConnectionConfig() (*SteampipeConfig, *modconfig.ErrorAndWarnings)

LoadConnectionConfig loads the connection config but not the workspace options this is called by the fdw

func LoadSteampipeConfig 

func LoadSteampipeConfig(modLocation string, commandName string) (*SteampipeConfig, *modconfig.ErrorAndWarnings)

LoadSteampipeConfig loads the HCL connection config and workspace options

func NewSteampipeConfig 

func NewSteampipeConfig(commandName string) *SteampipeConfig

func (*SteampipeConfig) ConfigMap 

func (c *SteampipeConfig) ConfigMap() map[string]interface{}

ConfigMap creates a config map to pass to viper

func (*SteampipeConfig) ConnectionList 

func (c *SteampipeConfig) ConnectionList() []*modconfig.Connection

func (*SteampipeConfig) ConnectionNames 

func (c *SteampipeConfig) ConnectionNames() []string

ConnectionNames returns a flat list of connection names

func (*SteampipeConfig) ConnectionsForPlugin 

func (c *SteampipeConfig) ConnectionsForPlugin(pluginLongName string, pluginVersion *version.Version) []*modconfig.Connection

func (*SteampipeConfig) GetConnectionOptions 

func (c *SteampipeConfig) GetConnectionOptions(connectionName string) *options.Connection

func (*SteampipeConfig) SetOptions 

func (c *SteampipeConfig) SetOptions(opts options.Options) (errorsAndWarnings *modconfig.ErrorAndWarnings)

func (*SteampipeConfig) String 

func (c *SteampipeConfig) String() string

func (*SteampipeConfig) Validate 

func (c *SteampipeConfig) Validate() (validationWarnings, validationErrors []string)

Validate validates all connections connections with validation errors are removed

type SteampipeNotification added in v0.20.0 

type SteampipeNotification struct {
	StructVersion int
	Type          PostgresNotificationType
}

func NewSchemaUpdateNotification added in v0.19.0 

func NewSchemaUpdateNotification(notificationType PostgresNotificationType) *SteampipeNotification

type ValidationFailure 

type ValidationFailure struct {
	Plugin             string
	ConnectionName     string
	Message            string
	ShouldDropIfExists bool
}

func (ValidationFailure) String 

func (v ValidationFailure) String() string

type VariableValidationFailedError added in v0.20.6 

type VariableValidationFailedError struct {
	// contains filtered or unexported fields
}

func (VariableValidationFailedError) Error added in v0.20.6 

func (m VariableValidationFailedError) Error() string

type WaitModeValue added in v0.20.0 

type WaitModeValue int
const (
	NoWait WaitModeValue = iota
	WaitForLoading
	WaitForReady
	WaitForSearchPath
)

type WorkspaceMetadata 

type WorkspaceMetadata struct {
	Id     string `json:"id,omitempty"`
	Handle string `json:"handle,omitempty"`
}

type WorkspaceProfileLoader added in v0.17.0 

type WorkspaceProfileLoader struct {
	DefaultProfile    *modconfig.WorkspaceProfile
	ConfiguredProfile *modconfig.WorkspaceProfile
	// contains filtered or unexported fields
}

func NewWorkspaceProfileLoader added in v0.17.0 

func NewWorkspaceProfileLoader(workspaceProfilePath string) (*WorkspaceProfileLoader, error)

func (*WorkspaceProfileLoader) GetActiveWorkspaceProfile added in v0.17.0 

func (l *WorkspaceProfileLoader) GetActiveWorkspaceProfile() *modconfig.WorkspaceProfile

Source Files

View all Source files

Directories

Path Synopsis
typeexpr
var_config

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.