Documentation ¶
Index ¶
Constants ¶
const ( HealthPollSeconds = 1 RaftHealthPollSeconds = 10 RecoveryPollSeconds = 1 ActiveNodeExpireSeconds = 5 BinlogFileHistoryDays = 1 MaintenanceOwner = "ham4db" AuditPageSize = 20 MaintenancePurgeDays = 7 TopologyMaxPoolConnections = 3 MaintenanceExpireMinutes = 10 AgentHttpTimeoutSeconds = 60 PseudoGTIDCoordinatesHistoryHeuristicMinutes = 2 DebugMetricsIntervalSeconds = 10 PseudoGTIDSchema = "_pseudo_gtid_" PseudoGTIDIntervalSeconds = 5 PseudoGTIDExpireMinutes = 60 StaleInstanceCoordinatesExpireSeconds = 60 CheckAutoPseudoGTIDGrantsIntervalSeconds = 60 SelectTrueQuery = "select 1" )
Variables ¶
var Config = newConfiguration()
Config is *the* configuration instance, used globally to get configuration data
Functions ¶
func MarkConfigurationLoaded ¶
func MarkConfigurationLoaded()
MarkConfigurationLoaded is called once configuration has first been loaded. Listeners on ConfigurationLoaded will get a notification
func ValidSecondsFromSeenToLastAttemptedCheck ¶
func ValidSecondsFromSeenToLastAttemptedCheck() uint
ValidSecondsFromSeenToLastAttemptedCheck returns the maximum allowed elapsed time between last_attempted_check_timestamp to last_checked_timestamp before we consider the instance as invalid.
func WaitForConfigurationToBeLoaded ¶
func WaitForConfigurationToBeLoaded()
WaitForConfigurationToBeLoaded does just that. It will return after the configuration file has been read off disk.
Types ¶
type Configuration ¶
type Configuration struct { Debug bool // set debug mode (similar to --debug option) EnableSyslog bool // Should logs be directed (in addition) to syslog daemon? AutoDiscover bool // flag,for investigate upstream and downstream instance ListenAddress string // Where ham4db HTTP should listen for TCP ListenSocket string // Where ham4db HTTP should listen for unix socket (default: empty; when given, TCP is disabled) HTTPAdvertise string // optional, for raft setups, what is the HTTP address this node will advertise to its peers (potentially use where behind NAT or when rerouting ports; example: "http://11.22.33.44:3030") AgentsServerPort string // port ham4db agents talk back to MysqlTopologyUser string MysqlTopologyPassword string MySQLTopologyCredentialsConfigFile string // my.cnf style configuration file from where to pick credentials. Expecting `user`, `password` under `[client]` section MySQLTopologySSLPrivateKeyFile string // Private key file used to authenticate with a Topology mysql instance with TLS MySQLTopologySSLCertFile string // Certificate PEM file used to authenticate with a Topology mysql instance with TLS MySQLTopologySSLCAFile string // Certificate Authority PEM file used to authenticate with a Topology mysql instance with TLS MySQLTopologySSLSkipVerify bool // If true, do not strictly validate mutual TLS certs for Topology mysql instances MySQLTopologyUseMutualTLS bool // Turn on TLS authentication with the Topology MySQL instances MySQLTopologyUseMixedTLS bool // Mixed TLS and non-TLS authentication with the Topology MySQL instances TLSCacheTTLFactor uint // Factor of InstancePollSeconds that we set as TLS info cache expiry BackendDB string // EXPERIMENTAL: type of backend db; either "mysql" or "sqlite3" SQLite3DataFile string // when BackendDB == "sqlite3", full path to sqlite3 datafile SkipDatabaseUpdate bool // When true, do not check backend database schema nor attempt to update it. Useful when you may be running multiple versions of ham4db, and you only wish certain boxes to dictate the db structure (or else any time a different ham4db version runs it will rebuild database schema) PanicIfDifferentDatabaseDeploy bool // When true, and this process finds the ham4db backend DB was provisioned by a different version, panic RaftEnabled bool // When true, setup ham4db in a raft consensus layout. When false (default) all Raft* variables are ignored RaftBind string RaftAdvertise string RaftDataDir string DefaultRaftPort int // if a RaftNodes entry does not specify port, use this one RaftNodes []string // Raft nodes to make initial connection with ExpectFailureAnalysisConcensus bool MaxPoolConnections int // The maximum size of the connection pool to the backend. BackendDatabase string BackendDBHost string BackendDBPort uint BackendDBUser string BackendDBPassword string MySQLCredentialsConfigFile string // my.cnf style configuration file from where to pick credentials. Expecting `user`, `password` under `[client]` section MySQLSSLPrivateKeyFile string // Private key file used to authenticate with the mysql instance with TLS MySQLSSLCertFile string // Certificate PEM file used to authenticate with the mysql instance with TLS MySQLSSLCAFile string // Certificate Authority PEM file used to authenticate with the mysql instance with TLS MySQLSSLSkipVerify bool // If true, do not strictly validate mutual TLS certs for the mysql instances MySQLUseMutualTLS bool // Turn on TLS authentication with the MySQL instance MySQLReadTimeoutSeconds int // Number of seconds before backend mysql read operation is aborted (driver-side) MySQLRejectReadOnly bool // Reject read only connections https://github.com/go-sql-driver/mysql#rejectreadonly ConnectTimeoutSeconds int // Number of seconds before connection is aborted (driver-side) MySQLDiscoveryReadTimeoutSeconds int // Number of seconds before topology mysql read operation is aborted (driver-side). Used for discovery queries. MySQLTopologyReadTimeoutSeconds int // Number of seconds before topology mysql read operation is aborted (driver-side). Used for all but discovery queries. ConnectionLifetimeSeconds int // Number of seconds the mysql driver will keep database connection alive before recycling it EnableAdaptor []string // plugin which enabled DefaultInstancePort int // In case port was not specified on command line SlaveLagQuery string // Synonym to ReplicationLagQuery ReplicationLagQuery string // custom query to check on replica lg (e.g. heartbeat table). Must return a single row with a single numeric column, which is the lag. ReplicationCredentialsQuery string // custom query to get replication credentials. Must return a single row, with two text columns: 1st is username, 2nd is password. This is optional, and can be used by ham4db to configure replication after master takeover or setup of co-masters. You need to ensure the ham4db user has the privileges to run this query DiscoverByShowSlaveHosts bool // Attempt SHOW SLAVE HOSTS before PROCESSLIST UseSuperReadOnly bool // Should ham4db super_read_only any time it sets read_only InstancePollSeconds uint // Number of seconds between instance reads InstanceWriteBufferSize int // Instance write buffer size (max number of instances to flush in one INSERT ODKU) BufferInstanceWrites bool // Set to 'true' for write-optimization on backend table (compromise: writes can be stale and overwrite non stale data) InstanceFlushIntervalMilliseconds int // Max interval between instance write buffer flushes SkipMaxScaleCheck bool // If you don't ever have MaxScale BinlogServer in your topology (and most people don't), set this to 'true' to save some pointless queries UnseenInstanceForgetHours uint // Number of hours after which an unseen instance is forgotten SnapshotTopologiesIntervalHours uint // Interval in hour between snapshot-topologies invocation. Default: 0 (disabled) DiscoveryMaxConcurrency uint // Number of goroutines doing hosts discovery DiscoveryQueueCapacity uint // Buffer size of the discovery queue. Should be greater than the number of DB instances being discovered DiscoveryQueueMaxStatisticsSize int // The maximum number of individual secondly statistics taken of the discovery queue DiscoveryCollectionRetentionSeconds uint // Number of seconds to retain the discovery collection information DiscoverySeeds []string // Hard coded array of type:hostname:port, ensuring ham4db discovers these hosts upon startup, assuming not already known to ham4db InstanceBulkOperationsWaitTimeoutSeconds uint // Time to wait on a single instance when doing bulk (many instances) operation HostnameResolveMethod string // Method by which to "normalize" hostname ("none"/"default"/"cname") MySQLHostnameResolveMethod string // Method by which to "normalize" hostname via MySQL server. ("none"/"@@hostname"/"@@report_host"; default "@@hostname") SkipBinlogServerUnresolveCheck bool // Skip the double-check that an unresolved hostname resolves back to same hostname for binlog servers ExpiryHostnameResolvesMinutes int // Number of minutes after which to expire hostname-resolves RejectHostnameResolvePattern string // Regexp pattern for resolved hostname that will not be accepted (not cached, not written to db). This is done to avoid storing wrong resolves due to network glitches. ReasonableReplicationLagSeconds int // Above this value is considered a problem ProblemIgnoreHostnameFilters []string // Will minimize problem visualization for hostnames matching given regexp filters VerifyReplicationFilters bool // Include replication filters check before approving topology refactoring ReasonableMaintenanceReplicationLagSeconds int // Above this value move-up and move-below are blocked CandidateInstanceExpireMinutes uint // Minutes after which a suggestion to use an instance as a candidate replica (to be preferably promoted on master failover) is expired. AuditLogFile string // Name of log file for audit operations. Disabled when empty. AuditToSyslog bool // If true, audit messages are written to syslog AuditToBackendDB bool // If true, audit messages are written to the backend DB's `audit` table (default: true) AuditPurgeDays uint // Days after which audit entries are purged from the database RemoveTextFromHostnameDisplay string // Text to strip off the hostname on cluster/clusters pages ReadOnly bool AuthenticationMethod string // Type of autherntication to use, if any. "" for none, "basic" for BasicAuth, "multi" for advanced BasicAuth, "proxy" for forwarded credentials via reverse proxy, "token" for token based access OAuthClientId string OAuthClientSecret string OAuthScopes []string HTTPAuthUser string // Username for HTTP Basic authentication (blank disables authentication) HTTPAuthPassword string // Password for HTTP Basic authentication AuthUserHeader string // HTTP header indicating auth user, when AuthenticationMethod is "proxy" PowerAuthUsers []string // On AuthenticationMethod == "proxy", list of users that can make changes. All others are read-only. PowerAuthGroups []string // list of unix groups the authenticated user must be a member of to make changes. AccessTokenUseExpirySeconds float32 // Time by which an issued token must be used AccessTokenExpiryMinutes float32 // Time after which HTTP access token expires ClusterNameToAlias map[string]string // map between regex matching cluster name to a human friendly alias DetectClusterAliasQuery string // Optional query (executed on topology instance) that returns the alias of a cluster. Query will only be executed on cluster master (though until the topology's master is resovled it may execute on other/all replicas). If provided, must return one row, one column DetectClusterDomainQuery string // Optional query (executed on topology instance) that returns the VIP/CNAME/Alias/whatever domain name for the master of this cluster. Query will only be executed on cluster master (though until the topology's master is resovled it may execute on other/all replicas). If provided, must return one row, one column DetectInstanceAliasQuery string // Optional query (executed on topology instance) that returns the alias of an instance. If provided, must return one row, one column DetectPromotionRuleQuery string // Optional query (executed on topology instance) that returns the promotion rule of an instance. If provided, must return one row, one column. DataCenterPattern string // Regexp pattern with one group, extracting the datacenter name from the hostname RegionPattern string // Regexp pattern with one group, extracting the region name from the hostname EnvironmentPattern string // Regexp pattern with one group, extracting environment info from hostname (e.g. combination of datacenter & prod/dev env) DetectDataCenterQuery string // Optional query (executed on topology instance) that returns the data center of an instance. If provided, must return one row, one column. Overrides DataCenterPattern and useful for installments where DC cannot be inferred by hostname DetectRegionQuery string // Optional query (executed on topology instance) that returns the region of an instance. If provided, must return one row, one column. Overrides RegionPattern and useful for installments where Region cannot be inferred by hostname DetectEnvironmentQuery string // Optional query (executed on topology instance) that returns the physical environment of an instance. If provided, must return one row, one column. Overrides EnvironmentPattern and useful for installments where env cannot be inferred by hostname DetectSemiSyncEnforcedQuery string // Optional query (executed on topology instance) to determine whether semi-sync is fully enforced for master writes (async fallback is not allowed under any circumstance). If provided, must return one row, one column, value 0 or 1. SupportFuzzyPoolHostnames bool // Should "submit-pool-instances" command be able to pass list of fuzzy instances (fuzzy means non-fqdn, but unique enough to recognize). Defaults 'true', implies more queries on backend db InstancePoolExpiryMinutes uint // Time after which entries in database_instance_pool are expired (resubmit via `submit-pool-instances`) PromotionIgnoreHostnameFilters []string // Will not promote replicas with hostname matching pattern (via -c recovery; for example, avoid promoting dev-dedicated machines) ServeAgentsHttp bool // Spawn another HTTP interface dedicated for ham4db-agent AgentsUseSSL bool // When "true" ham4db will listen on agents port with SSL as well as connect to agents via SSL AgentsUseMutualTLS bool // When "true" Use mutual TLS for the server to agent communication AgentSSLSkipVerify bool // When using SSL for the Agent, should we ignore SSL certification error AgentSSLPrivateKeyFile string // Name of Agent SSL private key file, applies only when AgentsUseSSL = true AgentSSLCertFile string // Name of Agent SSL certification file, applies only when AgentsUseSSL = true AgentSSLCAFile string // Name of the Agent Certificate Authority file, applies only when AgentsUseSSL = true AgentSSLValidOUs []string // Valid organizational units when using mutual TLS to communicate with the agents UseSSL bool // Use SSL on the server web port UseMutualTLS bool // When "true" Use mutual TLS for the server's web and API connections SSLSkipVerify bool // When using SSL, should we ignore SSL certification error SSLPrivateKeyFile string // Name of SSL private key file, applies only when UseSSL = true SSLCertFile string // Name of SSL certification file, applies only when UseSSL = true SSLCAFile string // Name of the Certificate Authority file, applies only when UseSSL = true SSLValidOUs []string // Valid organizational units when using mutual TLS StatusEndpoint string // Override the status endpoint. Defaults to '/api/status' StatusOUVerify bool // If true, try to verify OUs when Mutual TLS is on. Defaults to false AgentPollMinutes uint // Minutes between agent polling UnseenAgentForgetHours uint // Number of hours after which an unseen agent is forgotten StaleSeedFailMinutes uint // Number of minutes after which a stale (no progress) seed is considered failed. SeedAcceptableBytesDiff int64 // Difference in bytes between seed source & target data size that is still considered as successful copy SeedWaitSecondsBeforeSend int64 // Number of seconds for waiting before start send data command on agent AutoPseudoGTID bool // Should ham4db automatically inject Pseudo-GTID entries to the masters PseudoGTIDPattern string // Pattern to look for in binary logs that makes for a unique entry (pseudo GTID). When empty, Pseudo-GTID based refactoring is disabled. PseudoGTIDPatternIsFixedSubstring bool // If true, then PseudoGTIDPattern is not treated as regular expression but as fixed substring, and can boost search time PseudoGTIDMonotonicHint string // subtring in Pseudo-GTID entry which indicates Pseudo-GTID entries are expected to be monotonically increasing DetectPseudoGTIDQuery string // Optional query which is used to authoritatively decide whether pseudo gtid is enabled on instance BinlogEventsChunkSize int // Chunk size (X) for SHOW BINLOG|RELAYLOG EVENTS LIMIT ?,X statements. Smaller means less locking and mroe work to be done SkipBinlogEventsContaining []string // When scanning/comparing binlogs for Pseudo-GTID, skip entries containing given texts. These are NOT regular expressions (would consume too much CPU while scanning binlogs), just substrings to find. ReduceReplicationAnalysisCount bool // When true, replication analysis will only report instances where possibility of handled problems is possible in the first place (e.g. will not report most leaf nodes, that are mostly uninteresting). When false, provides an entry for every known instance FailureDetectionPeriodBlockMinutes int // The time for which an instance's failure discovery is kept "active", so as to avoid concurrent "discoveries" of the instance's failure; this preceeds any recovery process, if any. RecoveryPeriodBlockMinutes int // (supported for backwards compatibility but please use newer `RecoveryPeriodBlockSeconds` instead) The time for which an instance's recovery is kept "active", so as to avoid concurrent recoveries on same instance as well as flapping RecoveryPeriodBlockSeconds int // (overrides `RecoveryPeriodBlockMinutes`) The time for which an instance's recovery is kept "active", so as to avoid concurrent recoveries on same instance as well as flapping RecoveryIgnoreHostnameFilters []string // Recovery analysis will completely ignore hosts matching given patterns RecoverMasterClusterFilters []string // Only do master recovery on clusters matching these regexp patterns (of course the ".*" pattern matches everything) RecoverIntermediateMasterClusterFilters []string // Only do IM recovery on clusters matching these regexp patterns (of course the ".*" pattern matches everything) ProcessesShellCommand string // Shell that executes command scripts OnFailureDetectionProcesses []string // Processes to execute when detecting a failover scenario (before making a decision whether to failover or not). May and should use some of these placeholders: {failureType}, {instanceType}, {isMaster}, {isCoMaster}, {failureDescription}, {command}, {failedHost}, {failureCluster}, {failureClusterAlias}, {failureClusterDomain}, {failedPort}, {successorHost}, {successorPort}, {successorAlias}, {countReplicas}, {replicaHosts}, {isDowntimed}, {autoMasterRecovery}, {autoIntermediateMasterRecovery} PreGracefulTakeoverProcesses []string // Processes to execute before doing a failover (aborting operation should any once of them exits with non-zero code; order of execution undefined). May and should use some of these placeholders: {failureType}, {instanceType}, {isMaster}, {isCoMaster}, {failureDescription}, {command}, {failedHost}, {failureCluster}, {failureClusterAlias}, {failureClusterDomain}, {failedPort}, {successorHost}, {successorPort}, {countReplicas}, {replicaHosts}, {isDowntimed} PreFailoverProcesses []string // Processes to execute before doing a failover (aborting operation should any once of them exits with non-zero code; order of execution undefined). May and should use some of these placeholders: {failureType}, {instanceType}, {isMaster}, {isCoMaster}, {failureDescription}, {command}, {failedHost}, {failureCluster}, {failureClusterAlias}, {failureClusterDomain}, {failedPort}, {countReplicas}, {replicaHosts}, {isDowntimed} PostFailoverProcesses []string // Processes to execute after doing a failover (order of execution undefined). May and should use some of these placeholders: {failureType}, {instanceType}, {isMaster}, {isCoMaster}, {failureDescription}, {command}, {failedHost}, {failureCluster}, {failureClusterAlias}, {failureClusterDomain}, {failedPort}, {successorHost}, {successorPort}, {successorAlias}, {countReplicas}, {replicaHosts}, {isDowntimed}, {isSuccessful}, {lostReplicas}, {countLostReplicas} PostUnsuccessfulFailoverProcesses []string // Processes to execute after a not-completely-successful failover (order of execution undefined). May and should use some of these placeholders: {failureType}, {instanceType}, {isMaster}, {isCoMaster}, {failureDescription}, {command}, {failedHost}, {failureCluster}, {failureClusterAlias}, {failureClusterDomain}, {failedPort}, {successorHost}, {successorPort}, {successorAlias}, {countReplicas}, {replicaHosts}, {isDowntimed}, {isSuccessful}, {lostReplicas}, {countLostReplicas} PostMasterFailoverProcesses []string // Processes to execute after doing a master failover (order of execution undefined). Uses same placeholders as PostFailoverProcesses PostIntermediateMasterFailoverProcesses []string // Processes to execute after doing a master failover (order of execution undefined). Uses same placeholders as PostFailoverProcesses PostGracefulTakeoverProcesses []string // Processes to execute after runnign a graceful master takeover. Uses same placeholders as PostFailoverProcesses PostTakeMasterProcesses []string // Processes to execute after a successful Take-Master event has taken place CoMasterRecoveryMustPromoteOtherCoMaster bool // When 'false', anything can get promoted (and candidates are prefered over others). When 'true', ham4db will promote the other co-master or else fail DetachLostSlavesAfterMasterFailover bool // synonym to DetachLostReplicasAfterMasterFailover DetachLostReplicasAfterMasterFailover bool // Should replicas that are not to be lost in master recovery (i.e. were more up-to-date than promoted replica) be forcibly detached ApplyMySQLPromotionAfterMasterFailover bool // Should ham4db take upon itself to apply MySQL master promotion: set read_only=0, detach replication, etc. PreventCrossDataCenterMasterFailover bool // When true (default: false), cross-DC master failover are not allowed, ham4db will do all it can to only fail over within same DC, or else not fail over at all. PreventCrossRegionMasterFailover bool // When true (default: false), cross-region master failover are not allowed, ham4db will do all it can to only fail over within same region, or else not fail over at all. MasterFailoverLostInstancesDowntimeMinutes uint // Number of minutes to downtime any server that was lost after a master failover (including failed master & lost replicas). 0 to disable MasterFailoverDetachSlaveMasterHost bool // synonym to MasterFailoverDetachReplicaMasterHost MasterFailoverDetachReplicaMasterHost bool // Should ham4db issue a detach-replica-master-host on newly promoted master (this makes sure the new master will not attempt to replicate old master if that comes back to life). Defaults 'false'. Meaningless if ApplyMySQLPromotionAfterMasterFailover is 'true'. FailMasterPromotionOnLagMinutes uint // when > 0, fail a master promotion if the candidate replica is lagging >= configured number of minutes. FailMasterPromotionIfSQLThreadNotUpToDate bool // when true, and a master failover takes place, if candidate master has not consumed all relay logs, promotion is aborted with error DelayMasterPromotionIfSQLThreadNotUpToDate bool // when true, and a master failover takes place, if candidate master has not consumed all relay logs, delay promotion until the sql thread has caught up PostponeSlaveRecoveryOnLagMinutes uint // Synonym to PostponeReplicaRecoveryOnLagMinutes PostponeReplicaRecoveryOnLagMinutes uint // On crash recovery, replicas that are lagging more than given minutes are only resurrected late in the recovery process, after master/IM has been elected and processes executed. Value of 0 disables this feature OSCIgnoreHostnameFilters []string // OSC replicas recommendation will ignore replica hostnames matching given patterns MonitorType []string // Monitor type GraphiteAddr string // Optional; address of graphite port. If supplied, metric will be written here GraphitePath string // Prefix for graphite path. May include {hostname} magic placeholder GraphiteConvertHostnameDotsToUnderscores bool // If true, then hostname's dots are converted to underscores before being used in graphite path GraphitePollSeconds int // Graphite writes interval. 0 disables. URLPrefix string // URL prefix to run ham4db on non-root web path, e.g. /ham4db to put it behind nginx. DiscoveryIgnoreReplicaHostnameFilters []string // Regexp filters to apply to prevent auto-discovering new replicas. Usage: unreachable servers due to firewalls, applications which trigger binlog dumps DiscoveryIgnoreMasterHostnameFilters []string // Regexp filters to apply to prevent auto-discovering a master. Usage: pointing your master temporarily to replicate seom data from external host DiscoveryIgnoreHostnameFilters []string // Regexp filters to apply to prevent discovering instances of any kind ConsulAddress string // Address where Consul HTTP api is found. Example: 127.0.0.1:8500 ConsulScheme string // Scheme (http or https) for Consul ConsulAclToken string // ACL token used to write to Consul KV ConsulCrossDataCenterDistribution bool // should ham4db automatically auto-deduce all consul DCs and write KVs in all DCs ConsulKVStoreProvider string // Consul KV store provider (consul or consul-txn), default: "consul" ZkAddress string // UNSUPPERTED YET. Address where (single or multiple) ZooKeeper servers are found, in `srv1[:port1][,srv2[:port2]...]` format. Default port is 2181. Example: srv-a,srv-b:12181,srv-c KVClusterMasterPrefix string // Prefix to use for clusters' masters entries in KV stores (internal, consul, ZK), default: "mysql/master" WebMessage string // If provided, will be shown on all web pages below the title bar MaxConcurrentReplicaOperations int // Maximum number of concurrent operations on replicas }
Configuration makes for ham4db configuration input, which can be provided by user via JSON formatted file. Some of the parameteres have reasonable default values, and some (like database credentials) are strictly expected from user.
func ForceRead ¶
func ForceRead(fileName string) *Configuration
ForceRead reads configuration from given file name or bails out if it fails
func Read ¶
func Read(configFileList ...string) *Configuration
Read read configuration from zero, either, some or all given files, in order of input. A file can override configuration provided in previous file.
func Reload ¶
func Reload(extraFileList ...string) *Configuration
Reload re-read configuration from last used files
func (*Configuration) IsMySQL ¶
func (this *Configuration) IsMySQL() bool
func (*Configuration) IsSQLite ¶
func (this *Configuration) IsSQLite() bool
func (*Configuration) ToJSONString ¶
func (this *Configuration) ToJSONString() string
ToJSONString will marshal this configuration as JSON