elastic

package
v0.0.0-...-4da4744 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 26, 2024 License: Apache-2.0 Imports: 17 Imported by: 6

Documentation

Index

Constants

View Source 
const (
	// RepoBaseName is a basename that is used to namespace all snapshot repos
	RepoBaseName = "automate-elasticsearch-data"
)

Variables

This section is empty.

Functions

func IsSnapshotNotFound 

func IsSnapshotNotFound(e error) bool

IsSnapshotNotFound returns true if the underlying error represents a snapshot not found error

func SnapshotNotFound 

func SnapshotNotFound(serviceName string, snapshotName string) error

SnapshotNotFound error constructor

Types

type AwsElasticsearchAuth 

type AwsElasticsearchAuth struct {
	Enable    bool   `mapstructure:"enable"`
	ESUrl     string `mapstructure:"es_url"`
	Region    string `mapstructure:"region"`
	AccessKey string `mapstructure:"access_key"`
	SecretKey string `mapstructure:"secret_key"`
}

AwsElasticsearchAuth tells us if we should sign the create repo request with the aws signing algorithm. We'll use the provided credentials if available, falling back to instance credentials

type BackupsConfig 

type BackupsConfig struct {
	Backend          string           `mapstructure:"backend"`
	VerifyRepo       bool             `mapstructure:"verify_repo"`
	FsBackupsConfig  FsBackupsConfig  `mapstructure:"fs"`
	S3BackupsConfig  S3BackupsConfig  `mapstructure:"s3"`
	GCSBackupsConfig GCSBackupsConfig `mapstructure:"gcs"`
}

BackupsConfig contains settings for Es snapshot repo type and options, as expressed in the service's toml config file

func (*BackupsConfig) BackupsDisabled 

func (c *BackupsConfig) BackupsDisabled() bool

BackupsDisabled returns true if the backend is set to disabled

type ClusterHealth 

type ClusterHealth struct {
	ClusterName         string `json:"cluster_name"`
	Status              string `json:"status"`
	NumberOfNodes       int    `json:"number_of_nodes"`
	NumberOfDataNodes   int    `json:"number_of_data_nodes"`
	ActivePrimaryShards int    `json:"active_primary_shards"`
	ActiveShards        int    `json:"active_shards"`
}

ClusterHealth contains data returned by Es's cluster health API. See also: https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html

type CreateSnapshotStatus 

type CreateSnapshotStatus struct {
	State              string
	ProgressPercentage float64
	Message            string
}

CreateSnapshotStatus holds status information about a (possibly in-progress) Es snapshot. It basically contains the same info as the &api.CreateSnapshotStatusResponse type but is implemented separately to keep API concerns from leaking into this package

type DeleteError 

type DeleteError struct {
	Failures []FailedDeletion
}

DeleteError occurs when DeleteDocumentsFromIndexByAge is able to successfully invoke the ES DeleteByQuery API and the request completes without error, but reports that it was not able to delete the requested documents.

func (DeleteError) Error 

func (f DeleteError) Error() string

type DiskStats 

type DiskStats struct {
	MountPoint     string
	TotalBytes     int64
	AvailableBytes int64
}

DiskStats captures the disk usage stats for each data path on an ES cluster node, as reported by ES

type Elastic 

type Elastic struct {
	// contains filtered or unexported fields
}

Elastic is the interface to this component.

func New 

func New(esURL string, opts ...NewElasticOpts) (*Elastic, error)

New connects to the provided ES server instance and returns an Elastic instance containing a client

func (*Elastic) AllIndices 

func (es *Elastic) AllIndices() ([]string, error)

AllIndices returns an array of all index names. This is a development aid.

func (*Elastic) CreateRepository 

func (es *Elastic) CreateRepository(ctx context.Context, name string) (bool, error)

CreateRepository creates an ES repository for backups in a pre-configured location that's compatible with our test es5 setup for this component (reference test/elastic/config/elasticsearch.yml for config details, and the 'start-es5' target in es-sidecar-service's Makefile). This is a development aid.

func (*Elastic) CreateSnapshot 

func (es *Elastic) CreateSnapshot(ctx context.Context, serviceName, snapshotName, multiIndexSpec string, bc *BackupsConfig) error

CreateSnapshot creates/updates a repo with appropriate settings for the BackupsConfig, then creates a snapshot with the given snapshotName containing indices matching the multiIndexSpec. See also: https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-index.html

func (*Elastic) CreateSnapshotRepository 

func (es *Elastic) CreateSnapshotRepository(ctx context.Context, repoName string, bc *BackupsConfig) error

CreateSnapshotRepository creates a snapshot repository for the given service, with type and settings as set in the backups config If the repo exists, it will be updated with any settings that have changed

func (*Elastic) CreateTimeNamedIndices 

func (es *Elastic) CreateTimeNamedIndices(ctx context.Context, start time.Time, baseName string, days int)

CreateTimeNamedIndices creates a series of indices based on the provided time and number of days. This is a development aid.

func (*Elastic) DailyIndicesFor 

func (es *Elastic) DailyIndicesFor(ctx context.Context, baseName string) ([]string, error)

DailyIndicesFor returns the indices starting with indexName

func (*Elastic) DeleteDocumentsFromIndexByAge 

func (es *Elastic) DeleteDocumentsFromIndexByAge(ctx context.Context, index string, olderThanDays int, customPurgeField string) error

DeleteDocumentsFromIndexByAge deletes all documents in the specified index that are older than olderThanDays. Unlike DeleteTimeSeriesIndicesByAge, it looks at the complete date-time in determining age. It expects that the indexed contains a mapping of field 'end_time' to date.

func (*Elastic) DeleteSnapshot 

func (es *Elastic) DeleteSnapshot(ctx context.Context, serviceName, snapshotName string) error

DeleteSnapshot takes a context, service name and snapshot name and deletes the snapshot.

func (*Elastic) DeleteTimeSeriesIndicesByAge 

func (es *Elastic) DeleteTimeSeriesIndicesByAge(ctx context.Context, baseName string, olderThanDays int) error

DeleteTimeSeriesIndicesByAge deletes all indices names as baseName-YYYY-mm-dd that are older than olderThanDays. It looks only at dates in the index name when determining age - it does not consider time or reference any dates in the individual documents contained by the indices.

func (*Elastic) GetCreateSnapshotStatus 

func (es *Elastic) GetCreateSnapshotStatus(ctx context.Context, serviceName, snapshotName string) (CreateSnapshotStatus, error)

GetCreateSnapshotStatus calls the Es snapshot status API and returns the result. It is used to track the progress of a snapshot asynchronously. See also https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

func (*Elastic) GetDiskStats 

func (es *Elastic) GetDiskStats(ctx context.Context) ([]HostDiskStats, error)

GetDiskStats returns a slice of disk usage detail from all active nodes in the ES cluster.

func (*Elastic) GetRestoreSnapshotStatus 

func (es *Elastic) GetRestoreSnapshotStatus(ctx context.Context, serviceName, snapshotName string) (*RestoreSnapshotStatus, error)

GetRestoreSnapshotStatus gives the status of a possibly in-progress snapshot restore operation.

func (*Elastic) MakeMeAnIndex 

func (es *Elastic) MakeMeAnIndex(ctx context.Context, name string) error

MakeMeAnIndex creates a new index with the given name. This is a development and testing aid. TODO - this could probable just replace createTimeSeriesIndex below.

func (*Elastic) RemoveSnapshotRepository 

func (es *Elastic) RemoveSnapshotRepository(ctx context.Context, repoName string, bc *BackupsConfig) error

RemoveSnapshotRepository deletes a snapshot repository of the type set in the backups config

func (*Elastic) RestoreSnapshot 

func (es *Elastic) RestoreSnapshot(ctx context.Context, serviceName, snapshotName string, bc *BackupsConfig) error

RestoreSnapshot restores the given snapshot of the given service.

type FailedDeletion 

type FailedDeletion struct {
	ID   string
	Type string
}

FailedDeletion represents a document that could not be deleted.

type FsBackupsConfig 

type FsBackupsConfig struct {
	RootLocation           string `mapstructure:"root_location"`
	MaxSnapshotBytesPerSec string `mapstructure:"max_snapshot_bytes_per_sec"`
	MaxRestoreBytesPerSec  string `mapstructure:"max_restore_bytes_per_sec"`
}

FsBackupsConfig represents the settings available for "fs" type Es snapshot repos

type GCSBackupsConfig 

type GCSBackupsConfig struct {
	BucketName             string `mapstructure:"name"`
	ClientName             string `mapstructure:"client"`
	BasePath               string `mapstructure:"base_path"`
	ChunkSize              string `mapstructure:"chunk_size"`
	Compress               bool   `mapstructure:"compress"`
	MaxRestoreBytesPerSec  string `mapstructure:"max_restore_bytes_per_sec"`
	MaxSnapshotBytesPerSec string `mapstructure:"max_snapshot_bytes_per_sec"`
	ReadOnly               bool   `mapstructure:"readonly"`
	ApplicationName        string `mapstructure:"application_name"`
}

GCSBackupsConfig represents the settings available for "gcs" type Es snapshot repos

type HostDiskStats 

type HostDiskStats struct {
	Host         string
	PerDiskStats []DiskStats
}

HostDiskStats captures a collection of PerDiskStats for a given ES node.

type NewElasticOpts 

type NewElasticOpts func(*newElasticOpt)

NewElasticOpts are optional constructor parameters for New

func WithAwsAuth 

func WithAwsAuth(awsAuth *AwsElasticsearchAuth) NewElasticOpts

type NotFoundError 

type NotFoundError struct {
}

NotFoundError occurs when ES reports the requested index to be not found.

func (NotFoundError) Error 

func (f NotFoundError) Error() string

type RestoreSnapshotStatus 

type RestoreSnapshotStatus struct {
	State              string
	ProgressPercentage float64
	Message            string
}

RestoreSnapshotStatus holds status information about a (possibly in-progress) Es snapshot restore. It basically contains the same info as the &api.RestoreSnapshotStatus type but is implemented separately to keep API concerns from leaking into this package

type S3BackupsConfig 

type S3BackupsConfig struct {
	BucketName             string               `mapstructure:"name"`
	ClientName             string               `mapstructure:"client"`
	BasePath               string               `mapstructure:"base_path"`
	Compress               bool                 `mapstructure:"compress"`
	ServerSideEncryption   string               `mapstructure:"server_side_encryption"`
	BufferSize             string               `mapstructure:"buffer_size"`
	CannedACL              string               `mapstructure:"canned_acl"`
	StorageClass           string               `mapstructure:"storage_class"`
	MaxSnapshotBytesPerSec string               `mapstructure:"max_snapshot_bytes_per_sec"`
	MaxRestoreBytesPerSec  string               `mapstructure:"max_restore_bytes_per_sec"`
	ChunkSize              string               `mapstructure:"chunk_size"`
	Region                 string               `mapstructure:"region"`
	RoleARN                string               `mapstructure:"role_arn"`
	Auth                   AwsElasticsearchAuth `mapstructure:"aws_auth"`
}

S3BackupsConfig represents the settings available for "s3" type Es snapshot repos

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.