helix

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 22, 2017 License: Apache-2.0, MIT Imports: 16 Imported by: 0

README

go-helix GoDoc Build Status Coverage Status

A Go implementation of Apache Helix.

Currently the participant part only, compatible with the Apache Helix Java controller.

Installation

go get -u github.com/uber-go/go-helix

Quick Start

Init participant
participant, fatalErrChan := NewParticipant(
	zap.NewNop(),
	tally.NoopScope,
	"localhost:2181", // Zookeeper connect string
	"test_app", // application identifier
	"test_cluster", // helix cluster name
	"test_resource", // helix resource name
	"localhost", // participant host name
	123, // participant port
)

processor := NewStateModelProcessor()
processor.AddTransition("OFFLINE", "ONLINE", func(m *model.Message) (err error) {
    partitionNum, err := p.getPartitionNum(m)
    if err != nil {
        return err
    }
    // add logic here to save the owned partition and/or perform any actions for going online
})
processor.AddTransition("ONLINE", "OFFLINE", func(m *model.Message) (err error) {
   partitionNum, err := p.getPartitionNum(m)
   if err != nil {
       return err
   }
   // add logic here to remove the owned partition and/or perform any actions for going offline
})

participant.RegisterStateModel("OnlineOffline", processor)

err := participant.Connect() // initialization is complete if err is nil

// listen to the fatal error channel and handle the error by
// 1. recreate participant from scratch and connect, or
// 2. quit the program and restart
fatalErr := <- fatalErrChan
Use participant

Use the saved partitions to see if the partition should be handled by the participant.

Disconnect participant
participant.Disconnect()

Development Status: Beta

The APIs are functional. We do not expect, but there's no guarantee that no breaking changes will be made.

Contributing

We encourage and support an active, healthy community of contributors — including you! Details are in the contribution guide and the code of conduct. The go-helix maintainers keep an eye on issues and pull requests, but you can also report any negative conduct to oss-conduct@uber.com. That email list is a private, safe space; even the go-helix maintainers don't have access, so don't hesitate to hold us to a high standard.

Released under the MIT License.

Documentation

Index

Constants

View Source 
const (
	StateModelNameOnlineOffline = "OnlineOffline"

	StateModelStateOnline  = "ONLINE"
	StateModelStateOffline = "OFFLINE"
	StateModelStateDropped = "DROPPED"

	TargetController = "CONTROLLER"

	MsgTypeStateTransition = "STATE_TRANSITION"
)

TODO: add all the shared constants here

View Source 
const (
	// TestClusterName is the name of cluster used in test
	TestClusterName = "test_cluster"
	// TestResource is the name of resource used in test
	TestResource = "test_resource"
)

Variables

View Source 
var (
	// ErrClusterNotSetup means the helix data structure in zookeeper /{CLUSTER_NAME}
	// is not correct or does not exist
	ErrClusterNotSetup = errors.New("cluster not setup")

	// ErrNodeAlreadyExists the zookeeper node exists when it is not expected to
	ErrNodeAlreadyExists = errors.New("node already exists in cluster")

	// ErrNodeNotExist the zookeeper node does not exist when it is expected to
	ErrNodeNotExist = errors.New("node does not exist in config for cluster")

	// ErrInstanceNotExist the instance of a cluster does not exist when it is expected to
	ErrInstanceNotExist = errors.New("node does not exist in instances for cluster")

	// ErrStateModelDefNotExist the state model definition is expected to exist in zookeeper
	ErrStateModelDefNotExist = errors.New("state model not exist in cluster")

	// ErrResourceExists the resource already exists in cluster and cannot be added again
	ErrResourceExists = errors.New("resource already exists in cluster")

	// ErrResourceNotExists the resource does not exists and cannot be removed
	ErrResourceNotExists = errors.New("resource not exists in cluster")
)
View Source 
var (
	// ErrorNilUpdatedData returns when updateFn returns nil *model.ZNRecord without error
	ErrorNilUpdatedData = errors.New("data accessor: updated data is nil")
)

Functions

func CreateRandomString 

func CreateRandomString() string

CreateRandomString creates a random with numeric characters

func GetRandomPort 

func GetRandomPort() int32

GetRandomPort returns random valid port number (1~65535)

func IdealStateKey 

func IdealStateKey(clusterName string, resourceName string) string

IdealStateKey returns path for ideal state of a given cluster and resource

Types

type Admin 

type Admin struct {
	// contains filtered or unexported fields
}

Admin handles the administration task for the Helix cluster. Many of the operations are mirroring the implementions documented at http://helix.apache.org/0.7.0-incubating-docs/Quickstart.html

func NewAdmin 

func NewAdmin(zkConnectString string) (*Admin, error)

NewAdmin instantiates Admin

func (Admin) AddCluster 

func (adm Admin) AddCluster(cluster string, recreateIfExists bool) bool

AddCluster add a cluster to Helix. As a result, a znode will be created in zookeeper root named after the cluster name, and corresponding data structures are populated under this znode. The cluster would be dropped and recreated if recreateIfExists is true

func (Admin) AddNode 

func (adm Admin) AddNode(cluster string, node string) error

AddNode is the internal implementation corresponding to command ./helix-admin.sh --zkSvr <ZookeeperServerAddress> --addNode <clusterName instanceId> node is in the form of host_port

func (Admin) AddResource 

func (adm Admin) AddResource(
	cluster string, resource string, partitions int, stateModel string) error

AddResource implements the helix-admin.sh --addResource ./helix-admin.sh --zkSvr localhost:2199 --addResource MYCLUSTER myDB 6 MasterSlave

func (Admin) DisableResource 

func (adm Admin) DisableResource(cluster string, resource string) error

DisableResource disables the specified resource in the cluster.

func (Admin) DropCluster 

func (adm Admin) DropCluster(cluster string) error

DropCluster removes a helix cluster from zookeeper. This will remove the znode named after the cluster name from the zookeeper root.

func (Admin) DropInstance 

func (adm Admin) DropInstance(cluster string, instance string) error

DropInstance removes a participating instance from the helix cluster

func (Admin) DropNode 

func (adm Admin) DropNode(cluster string, node string) error

DropNode removes a node from a cluster. The corresponding znodes in zookeeper will be removed.

func (Admin) DropResource 

func (adm Admin) DropResource(cluster string, resource string) error

DropResource removes the specified resource from the cluster.

func (Admin) EnableResource 

func (adm Admin) EnableResource(cluster string, resource string) error

EnableResource enables the specified resource in the cluster

func (Admin) GetConfig 

func (adm Admin) GetConfig(
	cluster string, scope string, builder []string) (map[string]interface{}, error)

GetConfig obtains the configuration value of a property, defined by a config scope

func (Admin) GetInstances 

func (adm Admin) GetInstances(cluster string) error

GetInstances prints out lists of instances

func (Admin) ListClusterInfo 

func (adm Admin) ListClusterInfo(cluster string) (string, error)

ListClusterInfo shows the existing resources and instances in the glaster

func (Admin) ListClusters 

func (adm Admin) ListClusters() (string, error)

ListClusters shows all Helix managed clusters in the connected zookeeper cluster

func (Admin) ListInstanceInfo 

func (adm Admin) ListInstanceInfo(cluster string, instance string) (string, error)

ListInstanceInfo shows detailed information of an inspace in the helix cluster

func (Admin) ListInstances 

func (adm Admin) ListInstances(cluster string) (string, error)

ListInstances shows a list of instances participating the cluster.

func (Admin) ListResources 

func (adm Admin) ListResources(cluster string) (string, error)

ListResources shows a list of resources managed by the helix cluster

func (Admin) SetConfig 

func (adm Admin) SetConfig(cluster string, scope string, properties map[string]string) error

SetConfig set the configuration values for the cluster, defined by the config scope

type BaseHelixTestSuite 

type BaseHelixTestSuite struct {
	zk.BaseZkTestSuite
	Admin *Admin
}

BaseHelixTestSuite can be embedded in any test suites that need to interact with the test zk server or helix admin. It provides ZK connect string and Helix admin references for convenient uses in parent suites.

func (*BaseHelixTestSuite) SetupSuite 

func (s *BaseHelixTestSuite) SetupSuite()

SetupSuite ensures zk server is up

func (*BaseHelixTestSuite) TearDownSuite 

func (s *BaseHelixTestSuite) TearDownSuite()

TearDownSuite disconnects zk if not done already

type DataAccessor 

type DataAccessor struct {
	// contains filtered or unexported fields
}

DataAccessor helps to interact with Helix Data Types like IdealState, LiveInstance, Message it mirrors org.apache.helix.HelixDataAccessor

func (*DataAccessor) CreateParticipantMsg 

func (a *DataAccessor) CreateParticipantMsg(instanceName string, msg *model.Message) error

CreateParticipantMsg creates a message for the participant

func (*DataAccessor) CurrentState 

func (a *DataAccessor) CurrentState(instanceName, session, resourceName string,
) (*model.CurrentState, error)

CurrentState helps get Helix property with type CurrentState

func (*DataAccessor) IdealState 

func (a *DataAccessor) IdealState(resourceName string) (*model.IdealState, error)

IdealState helps get Helix property with type IdealState

func (*DataAccessor) InstanceConfig 

func (a *DataAccessor) InstanceConfig(path string) (*model.InstanceConfig, error)

InstanceConfig helps get Helix property with type Message

func (*DataAccessor) LiveInstance 

func (a *DataAccessor) LiveInstance(instanceName string) (*model.LiveInstance, error)

LiveInstance returns Helix property with type LiveInstance

func (*DataAccessor) Msg 

func (a *DataAccessor) Msg(path string) (*model.Message, error)

Msg helps get Helix property with type Message

func (*DataAccessor) StateModelDef 

func (a *DataAccessor) StateModelDef(stateModel string) (*model.StateModelDef, error)

StateModelDef helps get Helix property with type StateModelDef

type KeyBuilder 

type KeyBuilder struct {
	// contains filtered or unexported fields
}

KeyBuilder generates Zookeeper paths Mirrors org.apache.helix.PropertyKey#Builder

type Participant 

type Participant interface {
	Connect() error
	Disconnect()
	IsConnected() bool
	RegisterStateModel(stateModelName string, processor *StateModelProcessor)
	DataAccessor() *DataAccessor
	InstanceName() string
	Process(e zk.Event)
}

Participant is the Helix participant

func NewParticipant 

func NewParticipant(
	logger *zap.Logger,
	scope tally.Scope,
	zkConnectString string,
	application string,
	clusterName string,
	resourceName string,
	host string,
	port int32,
) (Participant, <-chan error)

NewParticipant instantiates a Participant, when an error is sent from the error chan, it means participant sees nonrecoverable errors user is expected to clean up and restart the program

type StateModel 

type StateModel interface {
	// GetState returns the state of a resource/partition combination
	GetState(resourceName string, partitionKey string) (string, bool)
	// UpdateState updates the state of a resource/partition combination
	UpdateState(resourceName string, partitionKey string, state string)
	// RemoveState removes the state of a resource/partition combination
	RemoveState(resourceName string, partitionKey string)
}

StateModel mirrors the behavior of org.apache.helix.participant.statemachine.StateModel and org.apache.helix.participant.statemachine.StateModelFactory it keeps the state of each resource/partition combination

func NewStateModel 

func NewStateModel() StateModel

NewStateModel creates a StateModel

type StateModelProcessor 

type StateModelProcessor struct {
	// fromState->toState->StateTransitionHandler
	Transitions map[string]map[string]StateTransitionHandler
}

StateModelProcessor handles state transitions This mirrors org.apache.helix.participant.statemachine.StateModelFactory

func NewStateModelProcessor 

func NewStateModelProcessor() *StateModelProcessor

NewStateModelProcessor functions similarly to StateMachineEngine

func (*StateModelProcessor) AddTransition 

func (p *StateModelProcessor) AddTransition(fromState string, toState string, handler StateTransitionHandler)

AddTransition adds a new transition handler

type StateTransitionHandler 

type StateTransitionHandler func(msg *model.Message) error

StateTransitionHandler is type for handler method

type TestParticipant 

type TestParticipant struct {
	Participant
}

TestParticipant is a participant used for test purpose

func NewTestParticipant 

func NewTestParticipant(
	logger *zap.Logger,
	scope tally.Scope,
	zkConnectString string,
	application string,
	clusterName string,
	resourceName string,
	host string,
	port int32,
) (*TestParticipant, <-chan error)

NewTestParticipant returns a TestParticipant

func (*TestParticipant) GetFatalErrorChan 

func (p *TestParticipant) GetFatalErrorChan() chan error

GetFatalErrorChan returns the fatal error chan, so user can send test error

type Transition 

type Transition struct {
	FromState string
	ToState   string
	Handler   StateTransitionHandler
}

Transition associates a handler function with state transitions

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.