Documentation ¶
Overview ¶
Package blip provides high-level data structs and const for integrating with Blip.
Index ¶
- Constants
- Variables
- func Bool(s string) bool
- func Debug(msg string, v ...interface{})
- func MonitorId(cfg ConfigMonitor) string
- func SetOrDefault(a, b string) string
- func StopLoss(v string) (uint, float64, error)
- func TimeLimit(p float64, d, max time.Duration) time.Duration
- func True(b *bool) bool
- type AWS
- type AWSConfigFactory
- type Collector
- type CollectorFactory
- type CollectorFactoryArgs
- type CollectorHelp
- type CollectorHelpError
- type CollectorHelpOption
- type CollectorKeyValue
- type CollectorMetric
- type Config
- type ConfigAPI
- type ConfigAWS
- type ConfigExporter
- type ConfigHTTP
- type ConfigHeartbeat
- type ConfigHighAvailability
- type ConfigMonitor
- type ConfigMonitorLoader
- type ConfigMonitorLoaderAWS
- type ConfigMonitorLoaderLocal
- type ConfigMySQL
- type ConfigPlanChange
- type ConfigPlans
- type ConfigSinks
- type ConfigStatePlan
- type ConfigTLS
- type DbFactory
- type Domain
- type Env
- type Factories
- type HTTPClientFactory
- type Level
- type MetricValue
- type Metrics
- type Plan
- type Plugins
- type Sink
- type SinkFactory
- type SinkFactoryArgs
Constants ¶
const ( UNKNOWN byte = iota CUMULATIVE_COUNTER DELTA_COUNTER GAUGE BOOL EVENT )
Metric types.
const ( STATE_NONE = "" STATE_OFFLINE = "offline" STATE_STANDBY = "standby" STATE_READ_ONLY = "read-only" STATE_ACTIVE = "active" )
Monitor states used for plan changing: https://cashapp.github.io/blip/plans/changing/
const ( DEFAULT_CONFIG_FILE = "blip.yaml" DEFAULT_DATABASE = "blip" )
const ( DEFAULT_MONITOR_USERNAME = "blip" DEFAULT_MONITOR_TIMEOUT_CONNECT = "10s" )
const ( EXPORTER_MODE_DUAL = "dual" // Blip and exporter run together EXPORTER_MODE_LEGACY = "legacy" // only exporter runs DEFAULT_EXPORTER_LISTEN_ADDR = "127.0.0.1:9104" DEFAULT_EXPORTER_PATH = "/metrics" DEFAULT_EXPORTER_PLAN = "default-exporter" )
const (
DEFAULT_API_BIND = "127.0.0.1:7522"
)
const (
DEFAULT_HEARTBEAT_TABLE = "blip.heartbeat"
)
const (
DEFAULT_PLANS_TABLE = "blip.plans"
)
const VERSION = "1.2.0"
Variables ¶
var (
Debugging = false
)
var ErrMore = errors.New("more metrics")
ErrMore signals that a collector will return more values. See https://cashapp.github.io/blip/develop/collectors/#long-running.
var FormatTime func(time.Time) string = func(t time.Time) string { return t.Format(time.RFC3339) }
var SHA = ""
Functions ¶
func MonitorId ¶
func MonitorId(cfg ConfigMonitor) string
func SetOrDefault ¶
SetOrDefault returns a if not empty, else it returns b. This is a convenience function to define variables with an explicit value or a DEFAULT_* value.
func TimeLimit ¶ added in v1.2.0
TimeLimit returns d minus p percentage (0.1 = 10%) of time up to max. For example, (0.1, 5s, 1s) returns 4.5s: 5000ms - 10% = 4500ms. But (0.1, 20s, 1s) returns 29s because 10% of 30s = 3s > 1s max, so the buffer is reduced to max. This is used to calculate engine max runtime (EMR) and collector max runtime (CMR).
func True ¶
True returns true if b is non-nil and true. This is convenience function related to *bool files in config structs, which is required for knowing when a bool config is explicitly set or not. If set, it's not changed; if not, it's set to the default value. That makes a good config experience but a less than ideal code experience because !*b will panic if b is nil, hence the need for this func.
Types ¶
type Collector ¶
type Collector interface { // Domain returns the Blip domain name. Domain() string // Help returns collector descipiton, options, and other usage printed by // blip --print-domains. Blip uses this information to validate user-provided // values in plans. Help() CollectorHelp // Prepare prepares a plan for future calls to Collect. The return function // is called once when the collector is destroyed; it allows the collector // to clean up. If Prepare returns an error, Blip will retry preparing the // plan. Therefore, Prepare should not retry on error (for example, if MySQL // is not online yet). Prepare(ctx context.Context, plan Plan) (func(), error) // Collect collects metrics for the previously prepared plan. Collect is only // called after Prepare returns nil. Collect(ctx context.Context, levelName string) ([]MetricValue, error) }
Collector collects metrics for a single metric domain.
type CollectorFactory ¶
type CollectorFactory interface {
Make(domain string, args CollectorFactoryArgs) (Collector, error)
}
A CollectorFactory makes one or more Collector.
type CollectorFactoryArgs ¶
type CollectorFactoryArgs struct { // Config is the full and final monitor config. Most collectors do not need // this, but some that collect metrics outside MySQL, like cloud metrics, // might need additional monitor config values. Config ConfigMonitor // DB is the connection to MySQL. It is safe for concurrent use, and it is // used concurrently by other parts of a monitor. The Collector must not // modify the connection, reconnect, and so forth--only use the connection. DB *sql.DB // MonitorId is the monitor identifier. The Collector must include // this value in all errors, output, and so forth. Everything monitor-related // in Blip is keyed on monitor ID. MonitorId string // Validate is true only when the plan loader is validating collectors. // Do not use this field. Validate bool }
CollectorFactoryArgs are provided by Blip to a CollectorFactory when making a Collector. The factory must use the args to create the collector.
type CollectorHelp ¶
type CollectorHelp struct { Domain string Description string Options map[string]CollectorHelpOption Errors map[string]CollectorHelpError Groups []CollectorKeyValue Meta []CollectorKeyValue Metrics []CollectorMetric }
Help represents information about a collector.
type CollectorHelpError ¶
type CollectorHelpOption ¶
type CollectorKeyValue ¶
type CollectorMetric ¶
type Config ¶
type Config struct { // Blip server API ConfigAPI `yaml:"api,omitempty"` HTTP ConfigHTTP `yaml:"http,omitempty"` MonitorLoader ConfigMonitorLoader `yaml:"monitor-loader,omitempty"` Sinks ConfigSinks `yaml:"sinks,omitempty"` // Monitor defaults AWS ConfigAWS `yaml:"aws,omitempty"` Exporter ConfigExporter `yaml:"exporter,omitempty"` HA ConfigHighAvailability `yaml:"ha,omitempty"` Heartbeat ConfigHeartbeat `yaml:"heartbeat,omitempty"` MySQL ConfigMySQL `yaml:"mysql,omitempty"` Plans ConfigPlans `yaml:"plans,omitempty"` Tags map[string]string `yaml:"tags,omitempty"` TLS ConfigTLS `yaml:"tls,omitempty"` Monitors []ConfigMonitor `yaml:"monitors,omitempty"` }
Config represents the Blip startup configuration.
func DefaultConfig ¶
func DefaultConfig() Config
func (*Config) ApplyDefaults ¶
func (*Config) InterpolateEnvVars ¶
func (c *Config) InterpolateEnvVars()
type ConfigAPI ¶
func DefaultConfigAPI ¶
func DefaultConfigAPI() ConfigAPI
func (*ConfigAPI) ApplyDefaults ¶
func (*ConfigAPI) InterpolateEnvVars ¶
func (c *ConfigAPI) InterpolateEnvVars()
type ConfigAWS ¶
type ConfigAWS struct { IAMAuth *bool `yaml:"iam-auth,omitempty"` PasswordSecret string `yaml:"password-secret,omitempty"` Region string `yaml:"region,omitempty"` DisableAutoRegion *bool `yaml:"disable-auto-region,omitempty"` DisableAutoTLS *bool `yaml:"disable-auto-tls,omitempty"` }
func DefaultConfigAWS ¶
func DefaultConfigAWS() ConfigAWS
func (*ConfigAWS) ApplyDefaults ¶
func (*ConfigAWS) InterpolateEnvVars ¶
func (c *ConfigAWS) InterpolateEnvVars()
func (*ConfigAWS) InterpolateMonitor ¶
func (c *ConfigAWS) InterpolateMonitor(m *ConfigMonitor)
type ConfigExporter ¶
type ConfigExporter struct { Flags map[string]string `yaml:"flags,omitempty"` Mode string `yaml:"mode,omitempty"` Plan string `yaml:"plan,omitempty"` }
func DefaultConfigExporter ¶
func DefaultConfigExporter() ConfigExporter
func (*ConfigExporter) ApplyDefaults ¶
func (c *ConfigExporter) ApplyDefaults(b Config)
func (*ConfigExporter) InterpolateEnvVars ¶
func (c *ConfigExporter) InterpolateEnvVars()
func (*ConfigExporter) InterpolateMonitor ¶
func (c *ConfigExporter) InterpolateMonitor(m *ConfigMonitor)
func (ConfigExporter) Validate ¶
func (c ConfigExporter) Validate() error
type ConfigHTTP ¶
type ConfigHTTP struct {
Proxy string `yaml:"proxy,omityempty"`
}
func DefaultConfigHTTP ¶
func DefaultConfigHTTP() ConfigHTTP
func (*ConfigHTTP) ApplyDefaults ¶
func (c *ConfigHTTP) ApplyDefaults(b Config)
func (*ConfigHTTP) InterpolateEnvVars ¶
func (c *ConfigHTTP) InterpolateEnvVars()
func (ConfigHTTP) Validate ¶
func (c ConfigHTTP) Validate() error
type ConfigHeartbeat ¶
type ConfigHeartbeat struct { Freq string `yaml:"freq,omitempty"` SourceId string `yaml:"source-id,omitempty"` Role string `yaml:"role,omitempty"` Table string `yaml:"table,omitempty"` }
func DefaultConfigHeartbeat ¶
func DefaultConfigHeartbeat() ConfigHeartbeat
func (*ConfigHeartbeat) ApplyDefaults ¶
func (c *ConfigHeartbeat) ApplyDefaults(b Config)
func (*ConfigHeartbeat) InterpolateEnvVars ¶
func (c *ConfigHeartbeat) InterpolateEnvVars()
func (*ConfigHeartbeat) InterpolateMonitor ¶
func (c *ConfigHeartbeat) InterpolateMonitor(m *ConfigMonitor)
func (ConfigHeartbeat) Validate ¶
func (c ConfigHeartbeat) Validate() error
type ConfigHighAvailability ¶
type ConfigHighAvailability struct{}
func DefaultConfigHA ¶
func DefaultConfigHA() ConfigHighAvailability
func (*ConfigHighAvailability) ApplyDefaults ¶
func (c *ConfigHighAvailability) ApplyDefaults(b Config)
func (*ConfigHighAvailability) InterpolateEnvVars ¶
func (c *ConfigHighAvailability) InterpolateEnvVars()
func (*ConfigHighAvailability) InterpolateMonitor ¶
func (c *ConfigHighAvailability) InterpolateMonitor(m *ConfigMonitor)
func (ConfigHighAvailability) Validate ¶
func (c ConfigHighAvailability) Validate() error
type ConfigMonitor ¶
type ConfigMonitor struct { MonitorId string `yaml:"id"` // ConfigMySQL: Socket string `yaml:"socket,omitempty"` Hostname string `yaml:"hostname,omitempty"` MyCnf string `yaml:"mycnf,omitempty"` Username string `yaml:"username,omitempty"` Password string `yaml:"password,omitempty"` PasswordFile string `yaml:"password-file,omitempty"` TimeoutConnect string `yaml:"timeout-connect,omitempty"` // Tags are passed to each metric sink. Tags inherit from config.tags, // but these monitor.tags take precedent (are not overwritten by config.tags). Tags map[string]string `yaml:"tags,omitempty"` AWS ConfigAWS `yaml:"aws,omitempty"` Exporter ConfigExporter `yaml:"exporter,omitempty"` HA ConfigHighAvailability `yaml:"ha,omitempty"` Heartbeat ConfigHeartbeat `yaml:"heartbeat,omitempty"` Plans ConfigPlans `yaml:"plans,omitempty"` Plan string `yaml:"plan,omitempty"` Sinks ConfigSinks `yaml:"sinks,omitempty"` TLS ConfigTLS `yaml:"tls,omitempty"` Meta map[string]string `yaml:"meta,omitempty"` }
func DefaultConfigMonitor ¶
func DefaultConfigMonitor() ConfigMonitor
func (*ConfigMonitor) ApplyDefaults ¶
func (c *ConfigMonitor) ApplyDefaults(b Config)
func (*ConfigMonitor) InterpolateEnvVars ¶
func (c *ConfigMonitor) InterpolateEnvVars()
func (*ConfigMonitor) InterpolateMonitor ¶
func (c *ConfigMonitor) InterpolateMonitor()
func (ConfigMonitor) Validate ¶
func (c ConfigMonitor) Validate() error
type ConfigMonitorLoader ¶
type ConfigMonitorLoader struct { Files []string `yaml:"files,omitempty"` StopLoss string `yaml:"stop-loss,omitempty"` AWS ConfigMonitorLoaderAWS `yaml:"aws,omitempty"` Local ConfigMonitorLoaderLocal `yaml:"local,omitempty"` }
func DefaultConfigMonitorLoader ¶
func DefaultConfigMonitorLoader() ConfigMonitorLoader
func (*ConfigMonitorLoader) ApplyDefaults ¶
func (c *ConfigMonitorLoader) ApplyDefaults(b Config)
func (*ConfigMonitorLoader) InterpolateEnvVars ¶
func (c *ConfigMonitorLoader) InterpolateEnvVars()
func (ConfigMonitorLoader) Validate ¶
func (c ConfigMonitorLoader) Validate() error
type ConfigMonitorLoaderAWS ¶
type ConfigMonitorLoaderAWS struct {
Regions []string `yaml:"regions,omitempty"`
}
func (ConfigMonitorLoaderAWS) Automatic ¶
func (c ConfigMonitorLoaderAWS) Automatic() bool
type ConfigMySQL ¶
type ConfigMySQL struct { Hostname string `yaml:"hostname,omitempty"` MyCnf string `yaml:"mycnf,omitempty"` Password string `yaml:"password,omitempty"` PasswordFile string `yaml:"password-file,omitempty"` Socket string `yaml:"socket,omitempty"` TimeoutConnect string `yaml:"timeout-connect,omitempty"` Username string `yaml:"username,omitempty"` }
ConfigMySQL are monitor defaults for each MySQL connection.
func DefaultConfigMySQL ¶
func DefaultConfigMySQL() ConfigMySQL
func (*ConfigMySQL) ApplyDefaults ¶
func (c *ConfigMySQL) ApplyDefaults(b Config)
func (*ConfigMySQL) InterpolateEnvVars ¶
func (c *ConfigMySQL) InterpolateEnvVars()
func (*ConfigMySQL) InterpolateMonitor ¶
func (c *ConfigMySQL) InterpolateMonitor(m *ConfigMonitor)
func (ConfigMySQL) Redacted ¶
func (c ConfigMySQL) Redacted() string
func (ConfigMySQL) Validate ¶
func (c ConfigMySQL) Validate() error
type ConfigPlanChange ¶
type ConfigPlanChange struct { Offline ConfigStatePlan `yaml:"offline,omitempty"` Standby ConfigStatePlan `yaml:"standby,omitempty"` ReadOnly ConfigStatePlan `yaml:"read-only,omitempty"` Active ConfigStatePlan `yaml:"active,omitempty"` }
func (*ConfigPlanChange) ApplyDefaults ¶
func (c *ConfigPlanChange) ApplyDefaults(b Config)
func (ConfigPlanChange) Enabled ¶
func (c ConfigPlanChange) Enabled() bool
func (*ConfigPlanChange) InterpolateEnvVars ¶
func (c *ConfigPlanChange) InterpolateEnvVars()
func (*ConfigPlanChange) InterpolateMonitor ¶
func (c *ConfigPlanChange) InterpolateMonitor(m *ConfigMonitor)
type ConfigPlans ¶
type ConfigPlans struct { Files []string `yaml:"files,omitempty"` Table string `yaml:"table,omitempty"` Monitor *ConfigMonitor `yaml:"monitor,omitempty"` Change ConfigPlanChange `yaml:"change,omitempty"` DisableDefaultPlans bool `yaml:"disable-default-plans"` }
func DefaultConfigPlans ¶
func DefaultConfigPlans() ConfigPlans
func (*ConfigPlans) ApplyDefaults ¶
func (c *ConfigPlans) ApplyDefaults(b Config)
func (*ConfigPlans) InterpolateEnvVars ¶
func (c *ConfigPlans) InterpolateEnvVars()
func (*ConfigPlans) InterpolateMonitor ¶
func (c *ConfigPlans) InterpolateMonitor(m *ConfigMonitor)
func (ConfigPlans) Validate ¶
func (c ConfigPlans) Validate() error
type ConfigSinks ¶
func DefaultConfigSinks ¶
func DefaultConfigSinks() ConfigSinks
func (ConfigSinks) ApplyDefaults ¶
func (c ConfigSinks) ApplyDefaults(b Config)
func (ConfigSinks) InterpolateEnvVars ¶
func (c ConfigSinks) InterpolateEnvVars()
func (ConfigSinks) InterpolateMonitor ¶
func (c ConfigSinks) InterpolateMonitor(m *ConfigMonitor)
func (ConfigSinks) Validate ¶
func (c ConfigSinks) Validate() error
type ConfigStatePlan ¶
type ConfigTLS ¶
type ConfigTLS struct { CA string `yaml:"ca,omitempty"` // ssl-ca Cert string `yaml:"cert,omitempty"` // ssl-cert Key string `yaml:"key,omitempty"` // ssl-key SkipVerify *bool `yaml:"skip-verify,omitempty"` Disable *bool `yaml:"disable,omitempty"` // ssl-mode from a my.cnf (see dbconn.ParseMyCnf) MySQLMode string `yaml:"-"` }
func DefaultConfigTLS ¶
func DefaultConfigTLS() ConfigTLS
func (*ConfigTLS) ApplyDefaults ¶
func (*ConfigTLS) InterpolateEnvVars ¶
func (c *ConfigTLS) InterpolateEnvVars()
func (*ConfigTLS) InterpolateMonitor ¶
func (c *ConfigTLS) InterpolateMonitor(m *ConfigMonitor)
type Domain ¶
type Domain struct { Name string `yaml:"-"` Metrics []string `yaml:"metrics,omitempty"` Options map[string]string `yaml:"options,omitempty"` Errors map[string]string `yaml:"errors,omitempty"` }
Domain is one metric domain for collecting related metrics.
type Env ¶
Env is the startup environment: command line args and environment variables. This is mostly used for testing to override the defaults.
type Factories ¶
type Factories struct { AWSConfig AWSConfigFactory DbConn DbFactory HTTPClient HTTPClientFactory }
Factories are interfaces that override certain object creation of Blip. Factories are optional, but if specified the override the built-in factories.
type HTTPClientFactory ¶
type Level ¶
type Level struct { Name string `yaml:"-"` Freq string `yaml:"freq"` Collect map[string]Domain `yaml:"collect"` }
Level is one collection frequency in a plan.
type MetricValue ¶
type MetricValue struct { // Name is the domain-specific metric name, like threads_running from the // status.global collector. Names are lowercase but otherwise not modified // (for example, hyphens and underscores are not changed). Name string // Value is the value of the metric. String values are not supported. // Boolean values are reported as 0 and 1. Value float64 // Type is the metric type: GAUGE, CUMULATIVE_COUNTER, and other const. Type byte // Group is the set of name-value pairs that determine the group to which // the metric value belongs. Only certain domains group metrics. Group map[string]string // Meta is optional key-value pairs that annotate or describe the metric value. Meta map[string]string }
MetricValue is one metric and its name, type, value, and tags. Tags are optional; the other fields are required and always set. This is the lowest-level data struct: a Collector reports metric values, which the monitor.Engine organize into Metrics by adding the appropriate metadata.
type Metrics ¶
type Metrics struct { Begin time.Time // when collection started End time.Time // when collection completed MonitorId string // ID of monitor (MySQL) Plan string // plan name Level string // level name Interval uint // interval number State string // state of monitor Values map[string][]MetricValue // keyed on domain }
Metrics are metrics collected for one plan level, from one MySQL instance.
type Plan ¶
type Plan struct { // Name is the name of the plan (required). // // When loaded from config.plans.files, Name is the exact name of the config. // The first file is the default plan if config.plan.default is not specified. // // When loaded from a config.plans.table, Name is the name column. The name // column cannot be NULL. The plan table is ordered by name (ascending) and // the first plan is the default if config.plan.default is not specified. // // config.plan.adjust.readonly and .active refer to Name. Name string // Levels are the collection frequencies that constitue the plan (required). Levels map[string]Level // MonitorId is the optional monitorId column from a plan table. // // When default plans are loaded from a table (config.plans.table), // the talbe is not filtered; all plans in the table are loaded. // // When a monitor (M) loads plans from a table (config.monitors.plans.table), // the table is filtered: WHERE monitorId = config.monitors.M.id. MonitorId string `yaml:"-"` // Source of plan: file name, table name, "plugin", or "blip" (internal plans). Source string `yaml:"-"` }
Plan represents different levels of metrics collection.
func (*Plan) InterpolateEnvVars ¶
func (p *Plan) InterpolateEnvVars()
func (*Plan) InterpolateMonitor ¶
func (p *Plan) InterpolateMonitor(mon *ConfigMonitor)
type Plugins ¶
type Plugins struct { // LoadConfig loads the Blip config on startup. It's passed the Blip default // config that should be applied like: // // mycfg.ApplyDefaults(def.DefaultConfig()) // // mycfg is the custom config loaded by the plugin, and def is the default // config passed to the plugin. Alternatively, the plugin can set values in // def (without unsetting default values). Without defaults, Blip might not // work as expected. // // Do not call InterpolateEnvVars. Blip calls that after loading the config. LoadConfig func(Config) (Config, error) // LoadMonitors loads monitors on startup and reloads them on POST /monitors/reload. LoadMonitors func(Config) ([]ConfigMonitor, error) // LoadPlans loads plans on startup. LoadPlans func(ConfigPlans) ([]Plan, error) // ModifyDB modifies the *sql.DB connection pool. Use with caution. ModifyDB func(*sql.DB, string) // StartMonitor allows a monitor to start by returning true. Else the monitor // is loaded but not started. This is used to load all monitors but start only // certain monitors. StartMonitor func(ConfigMonitor) bool // TransformMetrics transforms metrics before they are sent to sinks. // If it returns an error, all metrics are dropped (not sent). The function // is shared and called concurrently by all monitors. Use Metrics.MonitorId // to determine the source of the metrics. Metrics are ordered by Metrics.Interval // should not be reordered if delta counters are used (doing so will result // in negative or incorrect values). Otherwise, the slice of metrics can be modified. TransformMetrics func([]*Metrics) error }
Plugins are function callbacks that override specific functionality of Blip. Plugins are optional, but if specified it overrides the built-in functionality.
type Sink ¶
type Sink interface { // Send sends metrics to the sink. It must respect the context timeout, if any. Send(context.Context, *Metrics) error // Name returns the sink name (lowercase). It is used for monitor status to // report sink errors, if any. Name() string }
Sink sends metrics to an external destination.
type SinkFactory ¶
type SinkFactory interface {
Make(SinkFactoryArgs) (Sink, error)
}
SinkFactory makes a Sink for a monitor.
Directories ¶
Path | Synopsis |
---|---|
bin
|
|
Package dbconn provides a Factory that makes *sql.DB connections to MySQL.
|
Package dbconn provides a Factory that makes *sql.DB connections to MySQL. |
Package event provides a simple event stream in lieu of standard logging.
|
Package event provides a simple event stream in lieu of standard logging. |
status.global
Package statusglobal provides the status.global metric domain collector.
|
Package statusglobal provides the status.global metric domain collector. |
Package monitor provides core Blip components that, together, monitor one MySQL instance.
|
Package monitor provides core Blip components that, together, monitor one MySQL instance. |
Package plan provides the Loader singleton that loads metric collection plans.
|
Package plan provides the Loader singleton that loads metric collection plans. |
Package prom provides Prometheus emulation and translation.
|
Package prom provides Prometheus emulation and translation. |
Package server provides the core runtime for Blip.
|
Package server provides the core runtime for Blip. |
Package status provides real-time instantaneous status of every Blip component.
|
Package status provides real-time instantaneous status of every Blip component. |
Package test provides helper functions for tests.
|
Package test provides helper functions for tests. |