base

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2022 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Index

Constants

View Source 
const (

	// From IANA Service Name and Transport Protocol Port Number Registry. See
	// https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=cockroachdb
	//
	// This is used for both RPC and SQL connections unless --sql-addr
	// is used on the command line and/or SQLAddr is set in the Config object.
	DefaultPort = "26257"

	// The default port for HTTP-for-humans.
	DefaultHTTPPort = "8080"

	// NetworkTimeout is the timeout used for network operations.
	NetworkTimeout = 3 * time.Second

	// DefaultCertsDirectory is the default value for the cert directory flag.
	DefaultCertsDirectory = "${HOME}/.cockroach-certs"

	// NB: this can't easily become a variable as the UI hard-codes it to 10s.
	// See https://sqlfmt/cockroach/issues/20310.
	DefaultMetricsSampleInterval = 10 * time.Second

	// DefaultDescriptorLeaseDuration is the default mean duration a
	// lease will be acquired for. The actual duration is jittered using
	// the jitter fraction. Jittering is done to prevent multiple leases
	// from being renewed simultaneously if they were all acquired
	// simultaneously.
	DefaultDescriptorLeaseDuration = 5 * time.Minute

	// DefaultDescriptorLeaseJitterFraction is the default factor
	// that we use to randomly jitter the lease duration when acquiring a
	// new lease and the lease renewal timeout.
	DefaultDescriptorLeaseJitterFraction = 0.05

	// DefaultDescriptorLeaseRenewalTimeout is the default time
	// before a lease expires when acquisition to renew the lease begins.
	DefaultDescriptorLeaseRenewalTimeout = time.Minute
)

Base config defaults.

View Source 
const (
	// DefaultTempStorageMaxSizeBytes is the default maximum budget
	// for temp storage.
	DefaultTempStorageMaxSizeBytes = 32 * 1024 * 1024 * 1024 /* 32GB */
	// DefaultInMemTempStorageMaxSizeBytes is the default maximum budget
	// for in-memory temp storages.
	DefaultInMemTempStorageMaxSizeBytes = 100 * 1024 * 1024 /* 100MB */
)
View Source 
const (
	// DefaultMaxClockOffset is the default maximum acceptable clock offset value.
	// On Azure, clock offsets between 250ms and 500ms are common. On AWS and GCE,
	// clock offsets generally stay below 250ms. See comments on Config.MaxOffset
	// for more on this setting.
	DefaultMaxClockOffset = 500 * time.Millisecond

	// DefaultTxnHeartbeatInterval is how often heartbeats are sent from the
	// transaction coordinator to a live transaction. These keep it from
	// being preempted by other transactions writing the same keys. If a
	// transaction fails to be heartbeat within 5x the heartbeat interval,
	// it may be aborted by conflicting txns.
	DefaultTxnHeartbeatInterval = 1 * time.Second

	// SlowRequestThreshold is the amount of time to wait before considering a
	// request to be "slow".
	SlowRequestThreshold = 15 * time.Second

	// ChunkRaftCommandThresholdBytes is the threshold in bytes at which
	// to chunk or otherwise limit commands being sent to Raft.
	ChunkRaftCommandThresholdBytes = 256 * 1000

	// HeapProfileDir is the directory name where the heap profiler stores profiles
	// when there is a potential OOM situation.
	HeapProfileDir = "heap_profiler"

	// GoroutineDumpDir is the directory name where the goroutine dumper
	// stores dump when one of the dump heuristics is triggered.
	GoroutineDumpDir = "goroutine_dump"

	// CPUProfileDir is the directory name where the CPU profile dumper
	// stores profiles when the periodic CPU profile dump is enabled.
	CPUProfileDir = "pprof_dump"

	// InflightTraceDir is the directory name where the job trace dumper stores traces
	// when a job opts in to dumping its execution traces.
	InflightTraceDir = "inflight_trace_dump"

	// MinRangeMaxBytes is the minimum value for range max bytes.
	MinRangeMaxBytes = 64 << 10 // 64 KB
)
View Source 
const AuxiliaryDir = "auxiliary"

AuxiliaryDir is the path of the auxiliary dir relative to an engine.Engine's root directory. It must not be changed without a proper migration.

View Source 
const MinimumStoreSize = 10 * 64 << 20

MinimumStoreSize is the smallest size in bytes that a store can have. This number is based on config's defaultZoneConfig's RangeMaxBytes, which is extremely stable. To avoid adding the dependency on config here, it is just hard coded to 640MiB.

View Source 
const TempStoreID = -1

TempStoreID is used as the store id for a temp pebble engine's log

Variables

View Source 
var CheckEnterpriseEnabled = func(_ *cluster.Settings, _ uuid.UUID, org, feature string) error {
	return errEnterpriseNotEnabled
}

CheckEnterpriseEnabled returns a non-nil error if the requested enterprise feature is not enabled, including information or a link explaining how to enable it.

This function is overridden by an init hook in CCL builds.

View Source 
var (
	// DefaultTestStoreSpec is just a single in memory store of 512 MiB
	// with no special attributes.
	DefaultTestStoreSpec = StoreSpec{
		InMemory: true,
		Size: SizeSpec{
			InBytes: 512 << 20,
		},
	}
)
View Source 
var LicenseTTL = metric.NewGauge(licenseTTLMetadata)

LicenseTTL is a metric gauge that measures the number of seconds until the current enterprise license (if any) expires.

View Source 
var LicenseType = func(st *cluster.Settings) (string, error) {
	return "OSS", nil
}

LicenseType returns what type of license the cluster is running with, or "OSS" if it is an OSS build.

This function is overridden by an init hook in CCL builds.

View Source 
var TestingIDContainer = func() *SQLIDContainer {
	var c NodeIDContainer
	sc := c.SwitchToSQLIDContainer()
	if err := sc.SetSQLInstanceID(context.Background(), 10); err != nil {
		panic(err)
	}
	return sc
}()

TestingIDContainer is an SQLIDContainer with hard-coded SQLInstanceID of 10.

View Source 
var UpdateMetricOnLicenseChange = func(
	ctx context.Context,
	st *cluster.Settings,
	metric *metric.Gauge,
	ts timeutil.TimeSource,
	stopper *stop.Stopper,
) error {
	return nil
}

UpdateMetricOnLicenseChange is a function that's called on startup in order to connect the enterprise license setting update to the prometheus metric provided as an argument.

Functions

func DefaultHistogramWindowInterval 

func DefaultHistogramWindowInterval() time.Duration

DefaultHistogramWindowInterval returns the default rotation window for histograms.

func DefaultRetryOptions 

func DefaultRetryOptions() retry.Options

DefaultRetryOptions should be used for retrying most network-dependent operations.

func EmergencyBallastFile 

func EmergencyBallastFile(pathJoin func(...string) string, dataDir string) string

EmergencyBallastFile returns the path (relative to a data directory) used for an emergency ballast file. The returned path must be stable across releases (eg, we cannot change these constants), otherwise we may duplicate ballasts.

func GetAbsoluteStorePath 

func GetAbsoluteStorePath(fieldName string, p string) (string, error)

GetAbsoluteStorePath takes a (possibly relative) and returns the absolute path. Returns an error if the path begins with '~' or Abs fails. 'fieldName' is used in error strings.

func LookupAddr 

func LookupAddr(ctx context.Context, resolver *net.Resolver, host string) (string, error)

LookupAddr resolves the given address/host to an IP address. If multiple addresses are resolved, it returns the first IPv4 address available if there is one, otherwise the first address.

func PreventedStartupFile 

func PreventedStartupFile(dir string) string

PreventedStartupFile is the filename (relative to 'dir') used for files that can block server startup.

func UpdateAddrs 

func UpdateAddrs(ctx context.Context, addr, advAddr *string, ln net.Addr) error

UpdateAddrs updates the listen and advertise port numbers with those found during the call to net.Listen().

After ValidateAddrs() the actual listen addr should be equal to the one requested; only the port number can change because of auto-allocation. We do check this equality here and report a warning if any discrepancy is found.

Types

type ClusterIDContainer 

type ClusterIDContainer struct {
	// contains filtered or unexported fields
}

ClusterIDContainer is used to share a single Cluster ID instance between multiple layers. It allows setting and getting the value. Once a value is set, the value cannot change.

The cluster ID is determined on startup as follows: - If there are existing stores, their cluster ID is used. - If the node is bootstrapping, a new UUID is generated. - Otherwise, it is determined via gossip with other nodes.

func (*ClusterIDContainer) Get 

func (c *ClusterIDContainer) Get() uuid.UUID

Get returns the current cluster ID; uuid.Nil if it is unset.

func (*ClusterIDContainer) Reset 

func (c *ClusterIDContainer) Reset(val uuid.UUID)

Reset changes the ClusterID regardless of the old value.

Should only be used in testing code.

func (*ClusterIDContainer) Set 

func (c *ClusterIDContainer) Set(ctx context.Context, val uuid.UUID)

Set sets the current cluster ID. If it is already set, the value must match.

func (*ClusterIDContainer) String 

func (c *ClusterIDContainer) String() string

String returns the cluster ID, or "?" if it is unset.

type Config 

type Config struct {
	// Insecure specifies whether to disable security checks throughout
	// the code base.
	// This is really not recommended.
	// See: https://sqlfmt/cockroach/issues/53404
	Insecure bool

	// AcceptSQLWithoutTLS, when set, makes it possible for SQL
	// clients to authenticate without TLS on a secure cluster.
	//
	// Authentication is, as usual, subject to the HBA configuration: in
	// the default case, password authentication is still mandatory.
	AcceptSQLWithoutTLS bool

	// SSLCAKey is used to sign new certs.
	SSLCAKey string
	// SSLCertsDir is the path to the certificate/key directory.
	SSLCertsDir string

	// User running this process. It could be the user under which
	// the server is running or the user passed in client calls.
	User security.SQLUsername

	// Addr is the address the server is listening on.
	Addr string

	// AdvertiseAddr is the address advertised by the server to other nodes
	// in the cluster. It should be reachable by all other nodes and should
	// route to an interface that Addr is listening on.
	AdvertiseAddr string

	// ClusterName is the name used as a sanity check when a node joins
	// an uninitialized cluster, or when an uninitialized node joins an
	// initialized cluster. The initial RPC handshake verifies that the
	// name matches on both sides. Once the cluster ID has been
	// negotiated on both sides, the cluster name is not used any more.
	ClusterName string

	// DisableClusterNameVerification, when set, alters the cluster name
	// verification to only verify that a non-empty cluster name on
	// both sides match. This is meant for use while rolling an
	// existing cluster into using a new cluster name.
	DisableClusterNameVerification bool

	// SplitListenSQL indicates whether to listen for SQL
	// clients on a separate address from RPC requests.
	SplitListenSQL bool

	// SQLAddr is the configured SQL listen address.
	// This is used if SplitListenSQL is set to true.
	SQLAddr string

	// SQLAdvertiseAddr is the advertised SQL address.
	// This is computed from SQLAddr if specified otherwise Addr.
	SQLAdvertiseAddr string

	// HTTPAddr is the configured HTTP listen address.
	HTTPAddr string

	// DisableTLSForHTTP, if set, disables TLS for the HTTP listener.
	DisableTLSForHTTP bool

	// HTTPAdvertiseAddr is the advertised HTTP address.
	// This is computed from HTTPAddr if specified otherwise Addr.
	HTTPAdvertiseAddr string

	// RPCHeartbeatInterval controls how often a Ping request is sent on peer
	// connections to determine connection health and update the local view
	// of remote clocks.
	RPCHeartbeatInterval time.Duration

	// Enables the use of an PTP hardware clock user space API for HLC current time.
	// This contains the path to the device to be used (i.e. /dev/ptp0)
	ClockDevicePath string

	// AutoInitializeCluster, if set, causes the server to bootstrap the
	// cluster. Note that if two nodes are started with this flag set
	// and also configured to join each other, each node will bootstrap
	// its own unique cluster and the join will fail.
	//
	// The flag exists mostly for the benefit of tests, and for
	// `cockroach start-single-node`.
	AutoInitializeCluster bool
}

Config is embedded by server.Config. A base config is not meant to be used directly, but embedding configs should call cfg.InitDefaults().

func (*Config) AdminURL 

func (cfg *Config) AdminURL() *url.URL

AdminURL returns the URL for the admin UI.

func (*Config) HTTPRequestScheme 

func (cfg *Config) HTTPRequestScheme() string

HTTPRequestScheme returns "http" or "https" based on the value of Insecure and DisableTLSForHTTP.

func (*Config) HistogramWindowInterval 

func (*Config) HistogramWindowInterval() time.Duration

HistogramWindowInterval is used to determine the approximate length of time that individual samples are retained in in-memory histograms. Currently, it is set to the arbitrary length of six times the Metrics sample interval.

The length of the window must be longer than the sampling interval due to issue #12998, which was causing histograms to return zero values when sampled because all samples had been evicted.

Note that this is only intended to be a temporary fix for the above issue, as our current handling of metric histograms have numerous additional problems. These are tracked in github issue #7896, which has been given a relatively high priority in light of recent confusion around histogram metrics. For more information on the issues underlying our histogram system and the proposed fixes, please see issue #7896.

func (*Config) InitDefaults 

func (cfg *Config) InitDefaults()

InitDefaults sets up the default values for a config. This is also used in tests to reset global objects.

func (*Config) ValidateAddrs 

func (cfg *Config) ValidateAddrs(ctx context.Context) error

ValidateAddrs controls the address fields in the Config object and "fills in" the blanks:

  • the host part of Addr and HTTPAddr is resolved to an IP address if specified (it stays blank if blank to mean "all addresses").
  • the host part of AdvertiseAddr is filled in if blank, either from Addr if non-empty or os.Hostname(). It is also checked for resolvability.
  • non-numeric port numbers are resolved to numeric.

The addresses fields must be guaranteed by the caller to either be completely empty, or have both a host part and a port part separated by a colon. In the latter case either can be empty to indicate it's left unspecified.

type ExternalIODirConfig 

type ExternalIODirConfig struct {
	// Disables the use of external HTTP endpoints.
	// This turns off http:// external storage as well as any custom
	// endpoints cloud storage implementations.
	DisableHTTP bool
	// Disables the use of implicit credentials when accessing external services.
	// Implicit credentials are obtained from the system environment.
	// This turns off implicit credentials, and requires the user to provide
	// necessary access keys.
	DisableImplicitCredentials bool

	// DisableOutbound disables the use of any external-io that dials out such as
	// to s3, gcs, or even `nodelocal` as it may need to dial another node.
	DisableOutbound bool

	// EnableNonAdminImplicitAndArbitraryOutbound removes the usual restriction to
	// only admin users placed on usage of node-granted access, such as to the
	// implicit auth via the machine account for the node or to arbitrary network
	// addresses (which are accessed from the node and might otherwise not be
	// reachable). Instead, all users can use implicit auth, http addresses and
	// configure custom endpoints. This should only be used if all users with SQL
	// access should have access to anything the node has access to.
	EnableNonAdminImplicitAndArbitraryOutbound bool
}

ExternalIODirConfig describes various configuration options pertaining to external storage implementations. TODO(adityamaru): Rename ExternalIODirConfig to ExternalIOConfig because it is now used to configure both ExternalStorage and KMS.

type JoinListType 

type JoinListType []string

JoinListType is a slice of strings that implements pflag's value interface.

func (*JoinListType) Set 

func (jls *JoinListType) Set(value string) error

Set adds a new value to the JoinListType. It is the important part of pflag's value interface.

func (JoinListType) String 

func (jls JoinListType) String() string

String returns a string representation of all the JoinListType. This is part of pflag's value interface.

func (*JoinListType) Type 

func (jls *JoinListType) Type() string

Type returns the underlying type in string form. This is part of pflag's value interface.

type ModuleTestingKnobs 

type ModuleTestingKnobs interface {
	// ModuleTestingKnobs is a dummy function.
	ModuleTestingKnobs()
}

ModuleTestingKnobs is an interface for testing knobs for a submodule.

type NodeIDContainer 

type NodeIDContainer struct {
	// contains filtered or unexported fields
}

NodeIDContainer is used to share a single roachpb.NodeID or SQLInstanceID instance between multiple layers. It allows setting and getting the value. Once a value is set, the value cannot change. Note: we plan to rename it to denote its generic nature, see https://sqlfmt/cockroach/pull/73309

func (*NodeIDContainer) Get 

func (n *NodeIDContainer) Get() roachpb.NodeID

Get returns the current node ID; 0 if it is unset.

Note that Get() returns a value of type roachpb.NodeID even though the container is configured to store SQL instance IDs. This is because components that call Get() do so in a context where the type distinction between NodeID and SQLInstanceID does not matter, and we benefit from using a single type instead of duplicating the code. See for example the `rpc` package, where server-to-server RPCs get addressed with server IDs regardless of whether they are KV nodes or SQL instances. See also: https://sqlfmt/cockroach/pull/73309

func (*NodeIDContainer) Reset 

func (n *NodeIDContainer) Reset(val roachpb.NodeID)

Reset changes the NodeID regardless of the old value.

Should only be used in testing code.

func (*NodeIDContainer) SafeValue 

func (n *NodeIDContainer) SafeValue()

SafeValue implements the redact.SafeValue interface.

func (*NodeIDContainer) Set 

func (n *NodeIDContainer) Set(ctx context.Context, val roachpb.NodeID)

Set sets the current node ID. If it is already set, the value must match.

func (*NodeIDContainer) String 

func (n *NodeIDContainer) String() string

String returns the node ID, or "?" if it is unset.

func (*NodeIDContainer) SwitchToSQLIDContainer 

func (n *NodeIDContainer) SwitchToSQLIDContainer() *SQLIDContainer

SwitchToSQLIDContainer changes a NodeIDContainer to become able to store SQL instance IDs. After it has been switched, the original container will report the SQL instance ID value as NodeID via its Get() method.

type RaftConfig 

type RaftConfig struct {
	// RaftTickInterval is the resolution of the Raft timer.
	RaftTickInterval time.Duration

	// RaftElectionTimeoutTicks is the number of raft ticks before the
	// previous election expires. This value is inherited by individual stores
	// unless overridden.
	RaftElectionTimeoutTicks int

	// RaftHeartbeatIntervalTicks is the number of ticks that pass between heartbeats.
	RaftHeartbeatIntervalTicks int

	// RangeLeaseRaftElectionTimeoutMultiplier specifies what multiple the leader
	// lease active duration should be of the raft election timeout.
	RangeLeaseRaftElectionTimeoutMultiplier float64
	// RangeLeaseRenewalFraction specifies what fraction the range lease renewal
	// duration should be of the range lease active time. For example, with a
	// value of 0.2 and a lease duration of 10 seconds, leases would be eagerly
	// renewed 8 seconds into each lease. A value of zero means use the default
	// and a value of -1 means never preemptively renew the lease. A value of 1
	// means always renew.
	RangeLeaseRenewalFraction float64

	// RaftLogTruncationThreshold controls how large a single Range's Raft log
	// can grow. When a Range's Raft log grows above this size, the Range will
	// begin performing log truncations.
	RaftLogTruncationThreshold int64

	// RaftProposalQuota controls the maximum aggregate size of Raft commands
	// that a leader is allowed to propose concurrently.
	//
	// By default, the quota is set to a fraction of the Raft log truncation
	// threshold. In doing so, we ensure all replicas have sufficiently up to
	// date logs so that when the log gets truncated, the followers do not need
	// non-preemptive snapshots. Changing this deserves care. Too low and
	// everything comes to a grinding halt, too high and we're not really
	// throttling anything (we'll still generate snapshots).
	RaftProposalQuota int64

	// RaftMaxUncommittedEntriesSize controls how large the uncommitted tail of
	// the Raft log can grow. The limit is meant to provide protection against
	// unbounded Raft log growth when quorum is lost and entries stop being
	// committed but continue to be proposed.
	RaftMaxUncommittedEntriesSize uint64

	// RaftMaxSizePerMsg controls the maximum aggregate byte size of Raft log
	// entries the leader will send to followers in a single MsgApp. Smaller
	// value lowers the raft recovery cost (during initial probing and after
	// message loss during normal operation). On the other hand, it limits the
	// throughput during normal replication.
	RaftMaxSizePerMsg uint64

	// RaftMaxCommittedSizePerReady controls the maximum aggregate byte size of
	// committed Raft log entries a replica will receive in a single Ready.
	RaftMaxCommittedSizePerReady uint64

	// RaftMaxInflightMsgs controls how many "inflight" MsgApps Raft will send
	// to a follower without hearing a response. The total number of Raft log
	// entries is a combination of this setting and RaftMaxSizePerMsg. The
	// current default settings provide for up to 4 MB of raft log to be sent
	// without acknowledgement. With an average entry size of 1 KB that
	// translates to ~4096 commands that might be executed in the handling of a
	// single raft.Ready operation.
	RaftMaxInflightMsgs int

	// Splitting a range which has a replica needing a snapshot results in two
	// ranges in that state. The delay configured here slows down splits when in
	// that situation (limiting to those splits not run through the split
	// queue). The most important target here are the splits performed by
	// backup/restore.
	//
	// -1 to disable.
	RaftDelaySplitToSuppressSnapshotTicks int
}

RaftConfig holds raft tuning parameters.

func (RaftConfig) NodeLivenessDurations 

func (cfg RaftConfig) NodeLivenessDurations() (livenessActive, livenessRenewal time.Duration)

NodeLivenessDurations computes durations for node liveness expiration and renewal based on a default multiple of Raft election timeout.

func (RaftConfig) RaftElectionTimeout 

func (cfg RaftConfig) RaftElectionTimeout() time.Duration

RaftElectionTimeout returns the raft election timeout, as computed from the tick interval and number of election timeout ticks.

func (RaftConfig) RangeLeaseActiveDuration 

func (cfg RaftConfig) RangeLeaseActiveDuration() time.Duration

RangeLeaseActiveDuration is the duration of the active period of leader leases requested.

func (RaftConfig) RangeLeaseDurations 

func (cfg RaftConfig) RangeLeaseDurations() (rangeLeaseActive, rangeLeaseRenewal time.Duration)

RangeLeaseDurations computes durations for range lease expiration and renewal based on a default multiple of Raft election timeout.

func (RaftConfig) RangeLeaseRenewalDuration 

func (cfg RaftConfig) RangeLeaseRenewalDuration() time.Duration

RangeLeaseRenewalDuration specifies a time interval at the end of the active lease interval (i.e. bounded to the right by the start of the stasis period) during which operations will trigger an asynchronous renewal of the lease.

func (RaftConfig) SentinelGossipTTL 

func (cfg RaftConfig) SentinelGossipTTL() time.Duration

SentinelGossipTTL is time-to-live for the gossip sentinel. The sentinel informs a node whether or not it's connected to the primary gossip network and not just a partition. As such it must expire fairly quickly and be continually re-gossiped as a connected gossip network is necessary to propagate liveness. The replica which is the lease holder of the first range gossips it.

func (*RaftConfig) SetDefaults 

func (cfg *RaftConfig) SetDefaults()

SetDefaults initializes unset fields.

type SQLIDContainer 

type SQLIDContainer NodeIDContainer

SQLIDContainer is a variant of NodeIDContainer that contains SQL instance IDs.

func NewSQLIDContainerForNode 

func NewSQLIDContainerForNode(nodeID *NodeIDContainer) *SQLIDContainer

NewSQLIDContainerForNode sets up a SQLIDContainer which serves the underlying NodeID as the SQL instance ID.

func (*SQLIDContainer) OptionalNodeID 

func (c *SQLIDContainer) OptionalNodeID() (roachpb.NodeID, bool)

OptionalNodeID returns the NodeID and true, if the former is exposed. Otherwise, returns zero and false.

func (*SQLIDContainer) OptionalNodeIDErr 

func (c *SQLIDContainer) OptionalNodeIDErr(issue int) (roachpb.NodeID, error)

OptionalNodeIDErr is like OptionalNodeID, but returns an error (referring to the optionally supplied GitHub issues) if the ID is not present.

func (*SQLIDContainer) SQLInstanceID 

func (c *SQLIDContainer) SQLInstanceID() SQLInstanceID

SQLInstanceID returns the wrapped SQLInstanceID.

func (*SQLIDContainer) SafeValue 

func (c *SQLIDContainer) SafeValue()

SafeValue implements the redact.SafeValue interface.

func (*SQLIDContainer) SetSQLInstanceID 

func (c *SQLIDContainer) SetSQLInstanceID(ctx context.Context, sqlInstanceID SQLInstanceID) error

SetSQLInstanceID sets the SQL instance ID. It returns an error if we attempt to set an instance ID when the nodeID has already been initialized.

func (*SQLIDContainer) String 

func (c *SQLIDContainer) String() string

type SQLInstanceID 

type SQLInstanceID int32

A SQLInstanceID is an ephemeral ID assigned to a running instance of the SQL server. This is distinct from a NodeID, which is a long-lived identifier assigned to a node in the KV layer which is unique across all KV nodes in the cluster and persists across restarts. Instead, a Instance is similar to a process ID from the unix world: an integer assigned to the SQL server on process start which is unique across all SQL server processes running on behalf of the tenant, while the SQL server is running.

func (SQLInstanceID) String 

func (s SQLInstanceID) String() string

type SizeSpec 

type SizeSpec struct {
	// InBytes is used for calculating free space and making rebalancing
	// decisions. Zero indicates that there is no maximum size. This value is not
	// actually used by the engine and thus not enforced.
	InBytes int64
	Percent float64
}

SizeSpec contains size in different kinds of formats supported by CLI(%age, bytes).

func NewSizeSpec 

func NewSizeSpec(
	field redact.SafeString, value string, bytesRange *intInterval, percentRange *floatInterval,
) (SizeSpec, error)

NewSizeSpec parses the string passed into a --size flag and returns a SizeSpec if it is correctly parsed.

func (*SizeSpec) Set 

func (ss *SizeSpec) Set(value string) error

Set adds a new value to the StoreSpecValue. It is the important part of pflag's value interface.

func (*SizeSpec) String 

func (ss *SizeSpec) String() string

String returns a string representation of the SizeSpec. This is part of pflag's value interface.

func (*SizeSpec) Type 

func (ss *SizeSpec) Type() string

Type returns the underlying type in string form. This is part of pflag's value interface.

type StorageConfig 

type StorageConfig struct {
	Attrs roachpb.Attributes
	// Dir is the data directory for the Pebble instance.
	Dir string
	// If true, creating the instance fails if the target directory does not hold
	// an initialized instance.
	//
	// Makes no sense for in-memory instances.
	MustExist bool
	// MaxSize is used for calculating free space and making rebalancing
	// decisions. Zero indicates that there is no maximum size.
	MaxSize int64
	// BallastSize is the amount reserved by a ballast file for manual
	// out-of-disk recovery.
	BallastSize int64
	// Settings instance for cluster-wide knobs.
	Settings *cluster.Settings
	// UseFileRegistry is true if the file registry is needed (eg: encryption-at-rest).
	// This may force the store version to versionFileRegistry if currently lower.
	UseFileRegistry bool
	// EncryptionOptions is a serialized protobuf set by Go CCL code and passed
	// through to C CCL code to set up encryption-at-rest.  Must be set if and
	// only if encryption is enabled, otherwise left empty.
	EncryptionOptions []byte
}

StorageConfig contains storage configs for all storage engine.

func (StorageConfig) IsEncrypted 

func (sc StorageConfig) IsEncrypted() bool

IsEncrypted returns whether the StorageConfig has encryption enabled.

type StoreIDContainer 

type StoreIDContainer struct {
	// contains filtered or unexported fields
}

StoreIDContainer is added as a logtag in the pebbleLogger's context. The storeID value is later set atomically. The storeID can only be set after engine creation because the storeID is determined only after the pebbleLogger's context is created.

func (*StoreIDContainer) Get 

func (s *StoreIDContainer) Get() int32

Get returns the current storeID; 0 if it is unset.

func (*StoreIDContainer) SafeValue 

func (s *StoreIDContainer) SafeValue()

SafeValue implements the redact.SafeValue interface.

func (*StoreIDContainer) Set 

func (s *StoreIDContainer) Set(ctx context.Context, val int32)

Set sets the current storeID. If it is already set, the value should match.

func (*StoreIDContainer) String 

func (s *StoreIDContainer) String() string

String returns "temp" for temp stores, and the storeID for main stores if they haven't been initialized. If a main store hasn't been initialized, then "?" is returned.

type StoreSpec 

type StoreSpec struct {
	Path        string
	Size        SizeSpec
	BallastSize *SizeSpec
	InMemory    bool
	Attributes  roachpb.Attributes
	// StickyInMemoryEngineID is a unique identifier associated with a given
	// store which will remain in memory even after the default Engine close
	// until it has been explicitly cleaned up by CleanupStickyInMemEngine[s]
	// or the process has been terminated.
	// This only applies to in-memory storage engine.
	StickyInMemoryEngineID string
	// UseFileRegistry is true if the "file registry" store version is desired.
	// This is set by CCL code when encryption-at-rest is in use.
	UseFileRegistry bool
	// RocksDBOptions contains RocksDB specific options using a semicolon
	// separated key-value syntax ("key1=value1; key2=value2").
	RocksDBOptions string
	// PebbleOptions contains Pebble-specific options in the same format as a
	// Pebble OPTIONS file but treating any whitespace as a newline:
	// (Eg, "[Options] delete_range_flush_delay=2s flush_split_bytes=4096")
	PebbleOptions string
	// EncryptionOptions is a serialized protobuf set by Go CCL code and passed
	// through to C CCL code to set up encryption-at-rest.  Must be set if and
	// only if encryption is enabled, otherwise left empty.
	EncryptionOptions []byte
}

StoreSpec contains the details that can be specified in the cli pertaining to the --store flag.

func NewStoreSpec 

func NewStoreSpec(value string) (StoreSpec, error)

NewStoreSpec parses the string passed into a --store flag and returns a StoreSpec if it is correctly parsed. There are four possible fields that can be passed in, comma separated:

  • path=xxx The directory in which to the rocks db instance should be located, required unless using a in memory storage.
  • type=mem This specifies that the store is an in memory storage instead of an on disk one. mem is currently the only other type available.
  • size=xxx The optional maximum size of the storage. This can be in one of a few different formats.
  • 10000000000 -> 10000000000 bytes
  • 20GB -> 20000000000 bytes
  • 20GiB -> 21474836480 bytes
  • 0.02TiB -> 21474836480 bytes
  • 20% -> 20% of the available space
  • 0.2 -> 20% of the available space
  • attrs=xxx:yyy:zzz A colon separated list of optional attributes.

Note that commas are forbidden within any field name or value.

func (StoreSpec) IsEncrypted 

func (ss StoreSpec) IsEncrypted() bool

IsEncrypted returns whether the StoreSpec has encryption enabled.

func (StoreSpec) PreventedStartupFile 

func (ss StoreSpec) PreventedStartupFile() string

PreventedStartupFile returns the path to a file which, if it exists, should prevent the server from starting up. Returns an empty string for in-memory engines.

func (StoreSpec) String 

func (ss StoreSpec) String() string

String returns a fully parsable version of the store spec.

type StoreSpecList 

type StoreSpecList struct {
	Specs []StoreSpec
	// contains filtered or unexported fields
}

StoreSpecList contains a slice of StoreSpecs that implements pflag's value interface.

func (StoreSpecList) PriorCriticalAlertError 

func (ssl StoreSpecList) PriorCriticalAlertError() (err error)

PriorCriticalAlertError attempts to read the PreventedStartupFile for each store directory and returns their contents as a structured error.

These files typically request operator intervention after a corruption event by preventing the affected node(s) from starting back up.

func (*StoreSpecList) Set 

func (ssl *StoreSpecList) Set(value string) error

Set adds a new value to the StoreSpecValue. It is the important part of pflag's value interface.

func (StoreSpecList) String 

func (ssl StoreSpecList) String() string

String returns a string representation of all the StoreSpecs. This is part of pflag's value interface.

func (*StoreSpecList) Type 

func (ssl *StoreSpecList) Type() string

Type returns the underlying type in string form. This is part of pflag's value interface.

type SubzoneID 

type SubzoneID uint32

SubzoneID represents a subzone within a zone. It's the subzone's index within the parent zone + 1; there's no subzone 0 so that 0 can be used as a sentinel.

func SubzoneIDFromIndex 

func SubzoneIDFromIndex(idx int) SubzoneID

SubzoneIDFromIndex turns a subzone's index within its parent zone into its SubzoneID.

func (SubzoneID) ToSubzoneIndex 

func (id SubzoneID) ToSubzoneIndex() int32

ToSubzoneIndex turns a SubzoneID into the index corresponding to the correct Subzone within the parent zone's Subzones slice.

type TempStorageConfig 

type TempStorageConfig struct {
	// InMemory specifies whether the temporary storage will remain
	// in-memory or occupy a temporary subdirectory on-disk.
	InMemory bool
	// Path is the filepath of the temporary subdirectory created for
	// the temp storage.
	Path string
	// Mon will be used by the temp storage to register all its capacity requests.
	// It can be used to limit the disk or memory that temp storage is allowed to
	// use. If InMemory is set, than this has to be a memory monitor; otherwise it
	// has to be a disk monitor.
	Mon *mon.BytesMonitor
	// Spec stores the StoreSpec this TempStorageConfig will use.
	Spec StoreSpec
	// Settings stores the cluster.Settings this TempStoreConfig will use.
	Settings *cluster.Settings
}

TempStorageConfig contains the details that can be specified in the cli pertaining to temp storage flags, specifically --temp-dir and --max-disk-temp-storage.

func DefaultTestTempStorageConfig 

func DefaultTestTempStorageConfig(st *cluster.Settings) TempStorageConfig

DefaultTestTempStorageConfig is the associated temp storage for DefaultTestStoreSpec that is in-memory. It has a maximum size of 100MiB.

func DefaultTestTempStorageConfigWithSize 

func DefaultTestTempStorageConfigWithSize(
	st *cluster.Settings, maxSizeBytes int64,
) TempStorageConfig

DefaultTestTempStorageConfigWithSize is the associated temp storage for DefaultTestStoreSpec that is in-memory with the customized maximum size.

func TempStorageConfigFromEnv 

func TempStorageConfigFromEnv(
	ctx context.Context,
	st *cluster.Settings,
	useStore StoreSpec,
	parentDir string,
	maxSizeBytes int64,
) TempStorageConfig

TempStorageConfigFromEnv creates a TempStorageConfig. If parentDir is not specified and the specified store is in-memory, then the temp storage will also be in-memory.

type TestClusterArgs 

type TestClusterArgs struct {
	// ServerArgs will be copied and potentially adjusted according to the
	// ReplicationMode for each constituent TestServer. Used for all the servers
	// not overridden in ServerArgsPerNode.
	ServerArgs TestServerArgs
	// ReplicationMode controls how replication is to be done in the cluster.
	ReplicationMode TestClusterReplicationMode
	// If true, nodes will be started in parallel. This is useful in
	// testing certain recovery scenarios, although it makes store/node
	// IDs unpredictable. Even in ParallelStart mode, StartTestCluster
	// waits for all nodes to start before returning.
	ParallelStart bool

	// ServerArgsPerNode override the default ServerArgs with the value in this
	// map. The map's key is an index within TestCluster.Servers. If there is
	// no entry in the map for a particular server, the default ServerArgs are
	// used.
	//
	// These are indexes: the key 0 corresponds to the first node.
	//
	// A copy of an entry from this map will be copied to each individual server
	// and potentially adjusted according to ReplicationMode.
	ServerArgsPerNode map[int]TestServerArgs
}

TestClusterArgs contains the parameters one can set when creating a test cluster. It contains a TestServerArgs instance which will be copied over to every server.

The zero value means "ReplicationAuto".

type TestClusterReplicationMode 

type TestClusterReplicationMode int

TestClusterReplicationMode represents the replication settings for a TestCluster. 

const (
	// ReplicationAuto means that ranges are replicated according to the
	// production default zone config. Replication is performed as in
	// production, by the replication queue.
	// If ReplicationAuto is used, StartTestCluster() blocks until the initial
	// ranges are fully replicated.
	ReplicationAuto TestClusterReplicationMode = iota
	// ReplicationManual means that the split, merge and replication queues of all
	// servers are stopped, and the test must manually control splitting, merging
	// and replication through the TestServer.
	// Note that the server starts with a number of system ranges,
	// all with a single replica on node 1.
	ReplicationManual
)

func (TestClusterReplicationMode) String 

func (i TestClusterReplicationMode) String() string

type TestServerArgs 

type TestServerArgs struct {
	// Knobs for the test server.
	Knobs TestingKnobs

	*cluster.Settings
	RaftConfig

	// PartOfCluster must be set if the TestServer is joining others in a cluster.
	// If not set (and hence the server is the only one in the cluster), the
	// default zone config will be overridden to disable all replication - so that
	// tests don't get log spam about ranges not being replicated enough. This
	// is always set to true when the server is started via a TestCluster.
	PartOfCluster bool

	// Listener (if nonempty) is the listener to use for all incoming RPCs.
	// If a listener is installed, it informs the RPC `Addr` used below. The
	// Server itself knows to close it out. This is useful for when a test wants
	// manual control over how the join flags (`JoinAddr`) are populated, and
	// installs listeners manually to know which addresses to point to.
	Listener net.Listener

	// Addr (if nonempty) is the RPC address to use for the test server.
	Addr string
	// SQLAddr (if nonempty) is the SQL address to use for the test server.
	SQLAddr string
	// TenantAddr is the tenant KV address to use for the test server. If this
	// is nil, the tenant server will be set up using a random port. If this
	// is the empty string, no tenant server will be set up.
	TenantAddr *string
	// HTTPAddr (if nonempty) is the HTTP address to use for the test server.
	HTTPAddr string
	// DisableTLSForHTTP if set, disables TLS for the HTTP interface.
	DisableTLSForHTTP bool

	// JoinAddr is the address of a node we are joining.
	//
	// If left empty and the TestServer is being added to a nonempty cluster, this
	// will be set to the address of the cluster's first node.
	JoinAddr string

	// StoreSpecs define the stores for this server. If you want more than
	// one store per node, populate this array with StoreSpecs each
	// representing a store. If no StoreSpecs are provided then a single
	// DefaultTestStoreSpec will be used.
	StoreSpecs []StoreSpec

	// Locality is optional and will set the server's locality.
	Locality roachpb.Locality

	// TempStorageConfig defines parameters for the temp storage used as
	// working memory for distributed operations and CSV importing.
	// If not initialized, will default to DefaultTestTempStorageConfig.
	TempStorageConfig TempStorageConfig

	// ExternalIODir is used to initialize field in cluster.Settings.
	ExternalIODir string

	// ExternalIODirConfig is used to initialize the same-named
	// field on the server.Config struct.
	ExternalIODirConfig ExternalIODirConfig

	// Fields copied to the server.Config.
	Insecure                    bool
	RetryOptions                retry.Options // TODO(tbg): make testing knob.
	SocketFile                  string
	ScanInterval                time.Duration
	ScanMinIdleTime             time.Duration
	ScanMaxIdleTime             time.Duration
	SSLCertsDir                 string
	TimeSeriesQueryWorkerMax    int
	TimeSeriesQueryMemoryBudget int64
	SQLMemoryPoolSize           int64
	CacheSize                   int64

	// By default, test servers have AutoInitializeCluster=true set in
	// their config. If NoAutoInitializeCluster is set, that behavior is disabled
	// and the test becomes responsible for initializing the cluster.
	NoAutoInitializeCluster bool

	// If set, this will be appended to the Postgres URL by functions that
	// automatically open a connection to the server. That's equivalent to running
	// SET DATABASE=foo, which works even if the database doesn't (yet) exist.
	UseDatabase string

	// If set, this will be configured in the test server to check connections
	// from other test servers and to report in the SQL introspection.
	ClusterName string

	// Stopper can be used to stop the server. If not set, a stopper will be
	// constructed and it can be gotten through TestServerInterface.Stopper().
	Stopper *stop.Stopper

	// If set, the recording of events to the event log tables is disabled.
	DisableEventLog bool

	// If set, web session authentication will be disabled, even if the server
	// is running in secure mode.
	DisableWebSessionAuthentication bool

	// IF set, the demo login endpoint will be enabled.
	EnableDemoLoginEndpoint bool

	// Tracer, if set, will be used by the Server for creating Spans.
	Tracer *tracing.Tracer
	// TracingDefault kicks in if Tracer is not set. It is passed to the Tracer
	// that will be created for the server.
	TracingDefault tracing.TracingMode
	// If set, a TraceDir is initialized at the provided path.
	TraceDir string

	// DisableSpanConfigs disables the use of the span configs infrastructure
	// (in favor of the gossiped system config span). It's equivalent to setting
	// COCKROACH_DISABLE_SPAN_CONFIGS, and is only intended for tests written
	// with the system config span in mind.
	//
	// TODO(irfansharif): Remove all uses of this when we rip out the system
	// config span.
	DisableSpanConfigs bool
}

TestServerArgs contains the parameters one can set when creating a test server. Notably, TestServerArgs are passed to serverutils.StartServer(). They're defined in base because they need to be shared between testutils/serverutils (test code) and server.TestServer (non-test code).

The zero value is suitable for most tests.

type TestTenantArgs 

type TestTenantArgs struct {
	TenantID roachpb.TenantID

	// Existing, if true, indicates an existing tenant, rather than a new tenant
	// to be created by StartTenant.
	Existing bool

	// Settings allows the caller to control the settings object used for the
	// tenant cluster.
	Settings *cluster.Settings

	// AllowSettingClusterSettings, if true, allows the tenant to set in-memory
	// cluster settings.
	AllowSettingClusterSettings bool

	// Stopper, if not nil, is used to stop the tenant manually otherwise the
	// TestServer stopper will be used.
	Stopper *stop.Stopper

	// TestingKnobs for the test server.
	TestingKnobs TestingKnobs

	// Test server starts with secure mode by default. When this is set to true
	// it will switch to insecure
	ForceInsecure bool

	// MemoryPoolSize is the amount of memory in bytes that can be used by SQL
	// clients to store row data in server RAM.
	MemoryPoolSize int64

	// TempStorageConfig is used to configure temp storage, which stores
	// ephemeral data when processing large queries.
	TempStorageConfig *TempStorageConfig

	// ExternalIODirConfig is used to initialize the same-named
	// field on the server.Config struct.
	ExternalIODirConfig ExternalIODirConfig

	// ExternalIODir is used to initialize the same-named field on
	// the params.Settings struct.
	ExternalIODir string

	// If set, this will be appended to the Postgres URL by functions that
	// automatically open a connection to the server. That's equivalent to running
	// SET DATABASE=foo, which works even if the database doesn't (yet) exist.
	UseDatabase string

	// Skip check for tenant existence when running the test.
	SkipTenantCheck bool

	// Locality is used to initialize the same-named field on the server.Config
	// struct.
	Locality roachpb.Locality

	// SSLCertsDir is a path to a custom certs dir. If empty, will use the default
	// embedded certs.
	SSLCertsDir string

	// StartingSQLPort, if it is non-zero, is added to the tenant ID in order to
	// determine the tenant's SQL port.
	StartingSQLPort int

	// StartingHTTPPort, if it is non-zero, is added to the tenant ID in order to
	// determine the tenant's HTTP port.
	StartingHTTPPort int

	// TracingDefault controls whether the tracing will be on or off by default.
	TracingDefault tracing.TracingMode

	// RPCHeartbeatInterval controls how often the tenant sends Ping requests.
	RPCHeartbeatInterval time.Duration

	// GoroutineDumpDirName is used to initialize the same named field on the
	// SQLServer.BaseConfig field. It is used as the directory name for
	// goroutine dumps using goroutinedumper. If set, this directory should
	// be cleaned up once the test completes.
	GoroutineDumpDirName string

	// HeapProfileDirName is used to initialize the same named field on the
	// SQLServer.BaseConfig field. It is the directory name for heap profiles using
	// heapprofiler. If empty, no heap profiles will be collected during the test.
	// If set, this directory should be cleaned up after the test completes.
	HeapProfileDirName string
}

TestTenantArgs are the arguments used when creating a tenant from a TestServer.

type TestingKnobs 

type TestingKnobs struct {
	Store                        ModuleTestingKnobs
	KVClient                     ModuleTestingKnobs
	RangeFeed                    ModuleTestingKnobs
	SQLExecutor                  ModuleTestingKnobs
	SQLLeaseManager              ModuleTestingKnobs
	SQLSchemaChanger             ModuleTestingKnobs
	SQLDeclarativeSchemaChanger  ModuleTestingKnobs
	SQLTypeSchemaChanger         ModuleTestingKnobs
	GCJob                        ModuleTestingKnobs
	PGWireTestingKnobs           ModuleTestingKnobs
	StartupMigrationManager      ModuleTestingKnobs
	DistSQL                      ModuleTestingKnobs
	SQLEvalContext               ModuleTestingKnobs
	NodeLiveness                 ModuleTestingKnobs
	Server                       ModuleTestingKnobs
	TenantTestingKnobs           ModuleTestingKnobs
	JobsTestingKnobs             ModuleTestingKnobs
	BackupRestore                ModuleTestingKnobs
	TTL                          ModuleTestingKnobs
	Streaming                    ModuleTestingKnobs
	MigrationManager             ModuleTestingKnobs
	IndexUsageStatsKnobs         ModuleTestingKnobs
	SQLStatsKnobs                ModuleTestingKnobs
	SpanConfig                   ModuleTestingKnobs
	SQLLivenessKnobs             ModuleTestingKnobs
	TelemetryLoggingKnobs        ModuleTestingKnobs
	DialerKnobs                  ModuleTestingKnobs
	ProtectedTS                  ModuleTestingKnobs
	CapturedIndexUsageStatsKnobs ModuleTestingKnobs
	AdmissionControl             ModuleTestingKnobs
}

TestingKnobs contains facilities for controlling various parts of the system for testing.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.