automation

package
v2.0.0-beta.1+incompat... Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 3, 2016 License: BSD-3-Clause Imports: 17 Imported by: 0

Documentation

Overview

Package automation contains code to execute high-level cluster operations (e.g. resharding) as a series of low-level operations (e.g. vtctl, shell commands, ...).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AddMissingTaskID 

func AddMissingTaskID(tc []*automationpb.TaskContainer, taskIDGenerator *IDGenerator)

AddMissingTaskID assigns a task id to each task in "tc".

func AddTask 

func AddTask(t *automationpb.TaskContainer, taskName string, parameters map[string]string)

AddTask adds a new task to an existing task container.

func CreateLoggerEventToBufferFunction 

func CreateLoggerEventToBufferFunction(output *bytes.Buffer) func(*logutilpb.Event)

CreateLoggerEventToBufferFunction returns a function to add LoggerEvent structs to a given buffer, one line per event. The buffer can be used to return a multi-line string with all events.

func ExecuteVtctl 

func ExecuteVtctl(ctx context.Context, server string, args []string) (string, error)

ExecuteVtctl runs vtctl using vtctlclient. The stream of Event messages is concatenated into one output string.

func ExecuteVtworker 

func ExecuteVtworker(ctx context.Context, server string, args []string) (string, error)

ExecuteVtworker runs vtworker using vtworkerclient. The stream of LoggerEvent messages is concatenated into one output string.

func MarkTaskFailed 

func MarkTaskFailed(t *automationpb.Task, output string, err error)

MarkTaskFailed marks the task as failed.

func MarkTaskSucceeded 

func MarkTaskSucceeded(t *automationpb.Task, output string)

MarkTaskSucceeded marks the task as done.

func NewTask 

func NewTask(taskName string, parameters map[string]string) *automationpb.Task

NewTask creates a new task protobuf message for "taskName" with "parameters".

func NewTaskContainer 

func NewTaskContainer() *automationpb.TaskContainer

NewTaskContainer creates an empty task container. Use AddTask() to add tasks to it.

func NewTaskContainerWithSingleTask 

func NewTaskContainerWithSingleTask(taskName string, parameters map[string]string) *automationpb.TaskContainer

NewTaskContainerWithSingleTask creates a new task container with exactly one task.

Types

type ClusterOperationInstance 

type ClusterOperationInstance struct {
	automationpb.ClusterOperation
	// contains filtered or unexported fields
}

ClusterOperationInstance is a runtime type which enhances the protobuf message "ClusterOperation" with runtime specific data. Unlike the protobuf message, the additional runtime data will not be part of a checkpoint. Methods of this struct are not thread-safe.

func NewClusterOperationInstance 

func NewClusterOperationInstance(clusterOpID string, initialTask *automationpb.TaskContainer, taskIDGenerator *IDGenerator) ClusterOperationInstance

NewClusterOperationInstance creates a new cluster operation instance with one initial task.

func (ClusterOperationInstance) Clone 

func (c ClusterOperationInstance) Clone() ClusterOperationInstance

Clone creates a deep copy of the inner protobuf. Other elements e.g. taskIDGenerator are not deep-copied.

func (*ClusterOperationInstance) InsertTaskContainers 

func (c *ClusterOperationInstance) InsertTaskContainers(newTaskContainers []*automationpb.TaskContainer, pos int)

InsertTaskContainers inserts "newTaskContainers" at pos in the current list of task containers. Existing task containers will be moved after the new task containers.

type CopySchemaShardTask 

type CopySchemaShardTask struct {
}

CopySchemaShardTask runs vtctl CopySchemaShard to copy the schema from one shard to another.

func (*CopySchemaShardTask) OptionalParameters 

func (t *CopySchemaShardTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*CopySchemaShardTask) RequiredParameters 

func (t *CopySchemaShardTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*CopySchemaShardTask) Run 

func (t *CopySchemaShardTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type HorizontalReshardingTask 

type HorizontalReshardingTask struct {
}

HorizontalReshardingTask is a cluster operation which allows to increase the number of shards.

func (*HorizontalReshardingTask) OptionalParameters 

func (t *HorizontalReshardingTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*HorizontalReshardingTask) RequiredParameters 

func (t *HorizontalReshardingTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*HorizontalReshardingTask) Run 

func (t *HorizontalReshardingTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type IDGenerator 

type IDGenerator struct {
	// contains filtered or unexported fields
}

IDGenerator generates unique task and cluster operation IDs.

func (*IDGenerator) GetNextID 

func (ig *IDGenerator) GetNextID() string

GetNextID returns an ID which wasn't returned before.

type MigrateServedFromTask 

type MigrateServedFromTask struct {
}

MigrateServedFromTask runs vtctl MigrateServedFrom to let vertically split out tables get served from the new destination keyspace.

func (*MigrateServedFromTask) OptionalParameters 

func (t *MigrateServedFromTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*MigrateServedFromTask) RequiredParameters 

func (t *MigrateServedFromTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*MigrateServedFromTask) Run 

func (t *MigrateServedFromTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type MigrateServedTypesTask 

type MigrateServedTypesTask struct {
}

MigrateServedTypesTask runs vtctl MigrateServedTypes to migrate a serving type from the source shard to the shards that it replicates to.

func (*MigrateServedTypesTask) OptionalParameters 

func (t *MigrateServedTypesTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*MigrateServedTypesTask) RequiredParameters 

func (t *MigrateServedTypesTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*MigrateServedTypesTask) Run 

func (t *MigrateServedTypesTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type RebuildKeyspaceGraphTask 

type RebuildKeyspaceGraphTask struct {
}

RebuildKeyspaceGraphTask runs vtctl RebuildKeyspaceGraph to migrate a serving type from the source shard to the shards that it replicates to.

func (*RebuildKeyspaceGraphTask) OptionalParameters 

func (t *RebuildKeyspaceGraphTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*RebuildKeyspaceGraphTask) RequiredParameters 

func (t *RebuildKeyspaceGraphTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*RebuildKeyspaceGraphTask) Run 

func (t *RebuildKeyspaceGraphTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type Scheduler 

type Scheduler struct {
	// contains filtered or unexported fields
}

Scheduler executes automation tasks and maintains the execution state.

func NewScheduler 

func NewScheduler() (*Scheduler, error)

NewScheduler creates a new instance.

func (*Scheduler) Checkpoint 

func (s *Scheduler) Checkpoint(clusterOp ClusterOperationInstance)

Checkpoint should be called every time the state of the cluster op changes. It is used to update the copy of the state in activeClusterOperations.

func (*Scheduler) EnqueueClusterOperation 

func (s *Scheduler) EnqueueClusterOperation(ctx context.Context, req *automationpb.EnqueueClusterOperationRequest) (*automationpb.EnqueueClusterOperationResponse, error)

EnqueueClusterOperation can be used to start a new cluster operation.

func (*Scheduler) GetClusterOperationDetails 

func (s *Scheduler) GetClusterOperationDetails(ctx context.Context, req *automationpb.GetClusterOperationDetailsRequest) (*automationpb.GetClusterOperationDetailsResponse, error)

GetClusterOperationDetails can be used to query the full details of active or finished operations.

func (*Scheduler) Run 

func (s *Scheduler) Run()

Run processes queued cluster operations.

func (*Scheduler) ShutdownAndWait 

func (s *Scheduler) ShutdownAndWait()

ShutdownAndWait shuts down the scheduler and waits infinitely until all pending cluster operations have finished.

type SplitCloneTask 

type SplitCloneTask struct {
}

SplitCloneTask runs SplitClone on a remote vtworker to split an existing shard.

func (*SplitCloneTask) OptionalParameters 

func (t *SplitCloneTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*SplitCloneTask) RequiredParameters 

func (t *SplitCloneTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*SplitCloneTask) Run 

func (t *SplitCloneTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type SplitDiffTask 

type SplitDiffTask struct {
}

SplitDiffTask runs SplitDiff on a remote vtworker to compare the old shard against its new split shards.

func (*SplitDiffTask) OptionalParameters 

func (t *SplitDiffTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*SplitDiffTask) RequiredParameters 

func (t *SplitDiffTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*SplitDiffTask) Run 

func (t *SplitDiffTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type Task 

type Task interface {
	// Run executes the task using the key/values from parameters.
	// "newTaskContainers" contains new tasks which the task can emit. They'll be inserted in the cluster operation directly after this task. It may be "nil".
	// "output" may be empty. It contains any text which maybe must e.g. to debug the task or show it in the UI.
	Run(parameters map[string]string) (newTaskContainers []*automationpb.TaskContainer, output string, err error)

	// RequiredParameters() returns a list of parameter keys which must be provided as input for Run().
	RequiredParameters() []string

	// OptionalParameters() returns a list of parameter keys which are optional input for Run().
	OptionalParameters() []string
}

Task implementations can be executed by the scheduler.

type VerticalSplitCloneTask 

type VerticalSplitCloneTask struct {
}

VerticalSplitCloneTask runs VerticalSplitClone on a remote vtworker to split out tables from an existing keyspace to a different keyspace.

func (*VerticalSplitCloneTask) OptionalParameters 

func (t *VerticalSplitCloneTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*VerticalSplitCloneTask) RequiredParameters 

func (t *VerticalSplitCloneTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*VerticalSplitCloneTask) Run 

func (t *VerticalSplitCloneTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type VerticalSplitDiffTask 

type VerticalSplitDiffTask struct {
}

VerticalSplitDiffTask runs VerticalSplitDiff on a remote vtworker to compare the split out tables against the source keyspace.

func (*VerticalSplitDiffTask) OptionalParameters 

func (t *VerticalSplitDiffTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*VerticalSplitDiffTask) RequiredParameters 

func (t *VerticalSplitDiffTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*VerticalSplitDiffTask) Run 

func (t *VerticalSplitDiffTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type VerticalSplitTask 

type VerticalSplitTask struct {
}

VerticalSplitTask is a cluster operation to split out specific tables of one keyspace to a different keyspace.

func (*VerticalSplitTask) OptionalParameters 

func (t *VerticalSplitTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*VerticalSplitTask) RequiredParameters 

func (t *VerticalSplitTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*VerticalSplitTask) Run 

func (t *VerticalSplitTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type WaitForFilteredReplicationTask 

type WaitForFilteredReplicationTask struct {
}

WaitForFilteredReplicationTask runs vtctl WaitForFilteredReplication to block until the destination master (i.e. the receiving side of the filtered replication) has caught up up to max_delay with the source shard.

func (*WaitForFilteredReplicationTask) OptionalParameters 

func (t *WaitForFilteredReplicationTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*WaitForFilteredReplicationTask) RequiredParameters 

func (t *WaitForFilteredReplicationTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*WaitForFilteredReplicationTask) Run 

func (t *WaitForFilteredReplicationTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.