server

package
v0.0.0-...-91ad568 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 10, 2017 License: Apache-2.0 Imports: 45 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	PDBuildTS = "None"
	PDGitHash = "None"
)

Version information.

Functions

func GetMembers 

func GetMembers(etcdClient *clientv3.Client) ([]*pdpb.Member, error)

GetMembers return a slice of Members.

func InitLogger 

func InitLogger(cfg *Config) error

InitLogger initalizes PD's logger.

func LogPDInfo 

func LogPDInfo()

LogPDInfo prints the PD version information.

func ParseUrls 

func ParseUrls(s string) ([]url.URL, error)

ParseUrls parse a string into multiple urls. Export for api.

func PrepareJoinCluster 

func PrepareJoinCluster(cfg *Config) error

PrepareJoinCluster sends MemberAdd command to PD cluster, and returns the initial configuration of the PD cluster.

TL;TR: The join functionality is safe. With data, join does nothing, w/o data 

and it is not a member of cluster, join does MemberAdd, it returns an
error if PD tries to join itself, missing data or join a duplicated PD.

Etcd automatically re-joins the cluster if there is a data directory. So first it checks if there is a data directory or not. If there is, it returns an empty string (etcd will get the correct configurations from the data directory.)

If there is no data directory, there are following cases:

  • A new PD joins an existing cluster. What join does: MemberAdd, MemberList, then generate initial-cluster.

  • A failed PD re-joins the previous cluster. What join does: return an error. (etcd reports: raft log corrupted, truncated, or lost?)

  • A deleted PD joins to previous cluster. What join does: MemberAdd, MemberList, then generate initial-cluster. (it is not in the member list and there is no data, so we can treat it as a new PD.)

If there is a data directory, there are following special cases:

  • A failed PD tries to join the previous cluster but it has been deleted during its downtime. What join does: return "" (etcd will connect to other peers and find that the PD itself has been removed.)

  • A deleted PD joins the previous cluster. What join does: return "" (as etcd will read data directory and find that the PD itself has been removed, so an empty string is fine.)

func PrintPDInfo 

func PrintPDInfo()

PrintPDInfo prints the PD version information without log info.

Types

type Config 

type Config struct {
	*flag.FlagSet `json:"-"`

	Version bool `json:"-"`

	ClientUrls          string `toml:"client-urls" json:"client-urls"`
	PeerUrls            string `toml:"peer-urls" json:"peer-urls"`
	AdvertiseClientUrls string `toml:"advertise-client-urls" json:"advertise-client-urls"`
	AdvertisePeerUrls   string `toml:"advertise-peer-urls" json:"advertise-peer-urls"`

	Name    string `toml:"name" json:"name"`
	DataDir string `toml:"data-dir" json:"data-dir"`

	InitialCluster      string `toml:"initial-cluster" json:"initial-cluster"`
	InitialClusterState string `toml:"initial-cluster-state" json:"initial-cluster-state"`

	// Join to an existing pd cluster, a string of endpoints.
	Join string `toml:"join" json:"join"`

	// LeaderLease time, if leader doesn't update its TTL
	// in etcd after lease time, etcd will expire the leader key
	// and other servers can campaign the leader again.
	// Etcd onlys support seoncds TTL, so here is second too.
	LeaderLease int64 `toml:"lease" json:"lease"`

	// Log level.
	LogLevel string `toml:"log-level" json:"log-level"`
	// Log file.
	LogFile string `toml:"log-file" json:"log-file"`

	// TsoSaveInterval is the interval to save timestamp.
	TsoSaveInterval typeutil.Duration `toml:"tso-save-interval" json:"tso-save-interval"`

	Metric metricutil.MetricConfig `toml:"metric" json:"metric"`

	Schedule ScheduleConfig `toml:"schedule" json:"schedule"`

	Replication ReplicationConfig `toml:"replication" json:"replication"`

	// QuotaBackendBytes Raise alarms when backend size exceeds the given quota. 0 means use the default quota.
	// the default size is 2GB, the maximum is 8GB.
	QuotaBackendBytes typeutil.ByteSize `toml:"quota-backend-bytes" json:"quota-backend-bytes"`
	// AutoCompactionRetention for mvcc key value store in hour. 0 means disable auto compaction.
	// the default retention is 1 hour
	AutoCompactionRetention int `toml:"auto-compaction-retention" json:"auto-compaction-retention"`
	// contains filtered or unexported fields
}

Config is the pd server configuration.

func NewConfig 

func NewConfig() *Config

NewConfig creates a new config.

func NewTestMultiConfig 

func NewTestMultiConfig(count int) []*Config

NewTestMultiConfig is only for test to create multiple pd configurations. Because pd-client also needs this, so export here.

func NewTestSingleConfig 

func NewTestSingleConfig() *Config

NewTestSingleConfig is only for test to create one pd. Because pd-client also needs this, so export here.

func (*Config) Parse 

func (c *Config) Parse(arguments []string) error

Parse parses flag definitions from the argument list.

func (*Config) String 

func (c *Config) String() string

type Filter 

type Filter interface {
	// Return true if the store should not be used as a source store.
	FilterSource(store *storeInfo) bool
	// Return true if the store should not be used as a target store.
	FilterTarget(store *storeInfo) bool
}

Filter is an interface to filter source and target store.

type Handler 

type Handler struct {
	// contains filtered or unexported fields
}

Handler is a helper to export methods to handle API/RPC requests.

func (*Handler) AddBalanceLeaderScheduler 

func (h *Handler) AddBalanceLeaderScheduler() error

AddBalanceLeaderScheduler adds a balance-leader-scheduler.

func (*Handler) AddEvictLeaderScheduler 

func (h *Handler) AddEvictLeaderScheduler(storeID uint64) error

AddEvictLeaderScheduler adds an evict-leader-scheduler.

func (*Handler) AddGrantLeaderScheduler 

func (h *Handler) AddGrantLeaderScheduler(storeID uint64) error

AddGrantLeaderScheduler adds a grant-leader-scheduler.

func (*Handler) AddScheduler 

func (h *Handler) AddScheduler(s Scheduler) error

AddScheduler adds a scheduler.

func (*Handler) AddShuffleLeaderScheduler 

func (h *Handler) AddShuffleLeaderScheduler() error

AddShuffleLeaderScheduler adds a shuffle-leader-scheduler.

func (*Handler) AddShuffleRegionScheduler 

func (h *Handler) AddShuffleRegionScheduler() error

AddShuffleRegionScheduler adds a shuffle-region-scheduler.

func (*Handler) AddTransferLeaderOperator 

func (h *Handler) AddTransferLeaderOperator(regionID uint64, storeID uint64) error

AddTransferLeaderOperator adds an operator to transfer leader to the store.

func (*Handler) AddTransferPeerOperator 

func (h *Handler) AddTransferPeerOperator(regionID uint64, fromStoreID, toStoreID uint64) error

AddTransferPeerOperator adds an operator to transfer peer.

func (*Handler) AddTransferRegionOperator 

func (h *Handler) AddTransferRegionOperator(regionID uint64, storeIDs map[uint64]struct{}) error

AddTransferRegionOperator adds an operator to transfer region to the stores.

func (*Handler) GetAdminOperators 

func (h *Handler) GetAdminOperators() ([]Operator, error)

GetAdminOperators returns the running admin operators.

func (*Handler) GetLeaderOperators 

func (h *Handler) GetLeaderOperators() ([]Operator, error)

GetLeaderOperators returns the running leader operators.

func (*Handler) GetOperator 

func (h *Handler) GetOperator(regionID uint64) (Operator, error)

GetOperator returns the region operator.

func (*Handler) GetOperators 

func (h *Handler) GetOperators() ([]Operator, error)

GetOperators returns the running operators.

func (*Handler) GetOperatorsOfKind 

func (h *Handler) GetOperatorsOfKind(kind ResourceKind) ([]Operator, error)

GetOperatorsOfKind returns the running operators of the kind.

func (*Handler) GetRegionOperators 

func (h *Handler) GetRegionOperators() ([]Operator, error)

GetRegionOperators returns the running region operators.

func (*Handler) GetSchedulers 

func (h *Handler) GetSchedulers() ([]string, error)

GetSchedulers returns all names of schedulers.

func (*Handler) RemoveOperator 

func (h *Handler) RemoveOperator(regionID uint64) error

RemoveOperator removes the region operator.

func (*Handler) RemoveScheduler 

func (h *Handler) RemoveScheduler(name string) error

RemoveScheduler removes a scheduler by name.

type IDAllocator 

type IDAllocator interface {
	Alloc() (uint64, error)
}

IDAllocator is the allocator to generate unique ID.

type LogEvent 

type LogEvent struct {
	ID     uint64     `json:"id"`
	Code   msgType    `json:"code"`
	Status statusType `json:"status"`

	SplitEvent struct {
		Region uint64 `json:"region"`
		Left   uint64 `json:"left"`
		Right  uint64 `json:"right"`
	} `json:"split_event,omitempty"`

	AddReplicaEvent struct {
		Region uint64 `json:"region"`
		Store  uint64 `json:"store"`
	} `json:"add_replica_event,omitempty"`

	RemoveReplicaEvent struct {
		Region uint64 `json:"region"`
		Store  uint64 `json:"store"`
	} `json:"remove_replica_event,omitempty"`

	TransferLeaderEvent struct {
		Region    uint64 `json:"region"`
		StoreFrom uint64 `json:"store_from"`
		StoreTo   uint64 `json:"store_to"`
	} `json:"transfer_leader_event,omitempty"`
}

LogEvent is operator log event.

type Operator 

type Operator interface {
	GetRegionID() uint64
	GetResourceKind() ResourceKind
	Do(region *RegionInfo) (*pdpb.RegionHeartbeatResponse, bool)
}

Operator is an interface to schedule region.

type RaftCluster 

type RaftCluster struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

RaftCluster is used for cluster config management. Raft cluster key format: cluster 1 -> /1/raft, value is metapb.Cluster cluster 2 -> /2/raft For cluster 1 store 1 -> /1/raft/s/1, value is metapb.Store region 1 -> /1/raft/r/1, value is metapb.Region

func (*RaftCluster) BuryStore 

func (c *RaftCluster) BuryStore(storeID uint64, force bool) error

BuryStore marks a store as tombstone in cluster. State transition: Case 1: Up -> Tombstone (if force is true); Case 2: Offline -> Tombstone.

func (*RaftCluster) FetchEvents 

func (c *RaftCluster) FetchEvents(key uint64, all bool) []LogEvent

FetchEvents fetches the operator events.

func (*RaftCluster) GetConfig 

func (c *RaftCluster) GetConfig() *metapb.Cluster

GetConfig gets config from cluster.

func (*RaftCluster) GetRegionByID 

func (c *RaftCluster) GetRegionByID(regionID uint64) (*metapb.Region, *metapb.Peer)

GetRegionByID gets region and leader peer by regionID from cluster.

func (*RaftCluster) GetRegionByKey 

func (c *RaftCluster) GetRegionByKey(regionKey []byte) (*metapb.Region, *metapb.Peer)

GetRegionByKey gets region and leader peer by region key from cluster.

func (*RaftCluster) GetRegionInfoByID 

func (c *RaftCluster) GetRegionInfoByID(regionID uint64) *RegionInfo

GetRegionInfoByID gets regionInfo by regionID from cluster.

func (*RaftCluster) GetRegionInfoByKey 

func (c *RaftCluster) GetRegionInfoByKey(regionKey []byte) *RegionInfo

GetRegionInfoByKey gets regionInfo by region key from cluster.

func (*RaftCluster) GetRegions 

func (c *RaftCluster) GetRegions() []*metapb.Region

GetRegions gets regions from cluster.

func (*RaftCluster) GetStore 

func (c *RaftCluster) GetStore(storeID uint64) (*metapb.Store, *StoreStatus, error)

GetStore gets store from cluster.

func (*RaftCluster) GetStores 

func (c *RaftCluster) GetStores() []*metapb.Store

GetStores gets stores from cluster.

func (*RaftCluster) RemoveStore 

func (c *RaftCluster) RemoveStore(storeID uint64) error

RemoveStore marks a store as offline in cluster. State transition: Up -> Offline.

type RegionInfo 

type RegionInfo struct {
	*metapb.Region
	Leader       *metapb.Peer
	DownPeers    []*pdpb.PeerStats
	PendingPeers []*metapb.Peer
}

RegionInfo record detail region info

func (*RegionInfo) GetDownPeer 

func (r *RegionInfo) GetDownPeer(peerID uint64) *metapb.Peer

GetDownPeer return the down peers with specified peer id

func (*RegionInfo) GetFollower 

func (r *RegionInfo) GetFollower() *metapb.Peer

GetFollower randomly return a follow peer

func (*RegionInfo) GetFollowers 

func (r *RegionInfo) GetFollowers() map[uint64]*metapb.Peer

GetFollowers return a map indicate the follow peers distributed

func (*RegionInfo) GetPeer 

func (r *RegionInfo) GetPeer(peerID uint64) *metapb.Peer

GetPeer return the peer with specified peer id

func (*RegionInfo) GetPendingPeer 

func (r *RegionInfo) GetPendingPeer(peerID uint64) *metapb.Peer

GetPendingPeer return the pending peer with specified peer id

func (*RegionInfo) GetStoreIds 

func (r *RegionInfo) GetStoreIds() map[uint64]struct{}

GetStoreIds return a map indicate the region distributed

func (*RegionInfo) GetStorePeer 

func (r *RegionInfo) GetStorePeer(storeID uint64) *metapb.Peer

GetStorePeer return the peer in specified store

func (*RegionInfo) RemoveStorePeer 

func (r *RegionInfo) RemoveStorePeer(storeID uint64)

RemoveStorePeer remove the peer in specified store

type Replication 

type Replication struct {
	// contains filtered or unexported fields
}

Replication provides some help to do replication.

func (*Replication) GetDistinctScore 

func (r *Replication) GetDistinctScore(stores []*storeInfo, other *storeInfo) float64

GetDistinctScore returns the score that the other is distinct from the stores. A higher score means the other store is more different from the existed stores.

func (*Replication) GetLocationLabels 

func (r *Replication) GetLocationLabels() []string

GetLocationLabels returns the location labels for each region

func (*Replication) GetMaxReplicas 

func (r *Replication) GetMaxReplicas() int

GetMaxReplicas returns the number of replicas for each region.

func (*Replication) SetMaxReplicas 

func (r *Replication) SetMaxReplicas(replicas int)

SetMaxReplicas set the replicas for each region.

type ReplicationConfig 

type ReplicationConfig struct {
	// MaxReplicas is the number of replicas for each region.
	MaxReplicas uint64 `toml:"max-replicas,omitempty" json:"max-replicas"`

	// The label keys specified the location of a store.
	// The placement priorities is implied by the order of label keys.
	// For example, ["zone", "rack"] means that we should place replicas to
	// different zones first, then to different racks if we don't have enough zones.
	LocationLabels typeutil.StringSlice `toml:"location-labels,omitempty" json:"location-labels"`
}

ReplicationConfig is the replication configuration.

type ResourceKind 

type ResourceKind int

ResourceKind distinguishes different kinds of resources.

type ScheduleConfig 

type ScheduleConfig struct {
	// If the snapshot count of one store is greater than this value,
	// it will never be used as a source or target store.
	MaxSnapshotCount uint64 `toml:"max-snapshot-count,omitempty" json:"max-snapshot-count"`
	// MaxStoreDownTime is the max duration after which
	// a store will be considered to be down if it hasn't reported heartbeats.
	MaxStoreDownTime typeutil.Duration `toml:"max-store-down-time,omitempty" json:"max-store-down-time"`
	// LeaderScheduleLimit is the max coexist leader schedules.
	LeaderScheduleLimit uint64 `toml:"leader-schedule-limit,omitempty" json:"leader-schedule-limit"`
	// RegionScheduleLimit is the max coexist region schedules.
	RegionScheduleLimit uint64 `toml:"region-schedule-limit,omitempty" json:"region-schedule-limit"`
	// ReplicaScheduleLimit is the max coexist replica schedules.
	ReplicaScheduleLimit uint64 `toml:"replica-schedule-limit,omitempty" json:"replica-schedule-limit"`
}

ScheduleConfig is the schedule configuration.

type Scheduler 

type Scheduler interface {
	GetName() string
	GetResourceKind() ResourceKind
	GetResourceLimit() uint64
	Prepare(cluster *clusterInfo) error
	Cleanup(cluster *clusterInfo)
	Schedule(cluster *clusterInfo) Operator
}

Scheduler is an interface to schedule resources.

type Selector 

type Selector interface {
	SelectSource(stores []*storeInfo, filters ...Filter) *storeInfo
	SelectTarget(stores []*storeInfo, filters ...Filter) *storeInfo
}

Selector is an interface to select source and target store to schedule.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server is the pd server.

func CreateServer 

func CreateServer(cfg *Config) *Server

CreateServer creates the UNINITIALIZED pd server with given configuration.

func NewServer 

func NewServer(cfg *Config) (*Server, error)

NewServer creates the pd server with given configuration.

func (*Server) AllocID 

func (s *Server) AllocID(ctx context.Context, request *pdpb.AllocIDRequest) (*pdpb.AllocIDResponse, error)

AllocID implements gRPC PDServer.

func (*Server) AskSplit 

func (s *Server) AskSplit(ctx context.Context, request *pdpb.AskSplitRequest) (*pdpb.AskSplitResponse, error)

AskSplit implements gRPC PDServer.

func (*Server) Bootstrap 

func (s *Server) Bootstrap(ctx context.Context, request *pdpb.BootstrapRequest) (*pdpb.BootstrapResponse, error)

Bootstrap implements gRPC PDServer.

func (*Server) Close 

func (s *Server) Close()

Close closes the server.

func (*Server) ClusterID 

func (s *Server) ClusterID() uint64

ClusterID returns the cluster ID of this server.

func (*Server) GetAddr 

func (s *Server) GetAddr() string

GetAddr returns the server urls for clients.

func (*Server) GetClient 

func (s *Server) GetClient() *clientv3.Client

GetClient returns builtin etcd client.

func (*Server) GetCluster 

func (s *Server) GetCluster() *metapb.Cluster

GetCluster gets cluster

func (*Server) GetClusterConfig 

func (s *Server) GetClusterConfig(ctx context.Context, request *pdpb.GetClusterConfigRequest) (*pdpb.GetClusterConfigResponse, error)

GetClusterConfig implements gRPC PDServer.

func (*Server) GetConfig 

func (s *Server) GetConfig() *Config

GetConfig gets the config information.

func (*Server) GetEndpoints 

func (s *Server) GetEndpoints() []string

GetEndpoints returns the etcd endpoints for outer use.

func (*Server) GetHandler 

func (s *Server) GetHandler() *Handler

GetHandler returns the handler for API.

func (*Server) GetLeader 

func (s *Server) GetLeader() (*pdpb.Member, error)

GetLeader gets pd cluster leader.

func (*Server) GetMembers 

func (s *Server) GetMembers(context.Context, *pdpb.GetMembersRequest) (*pdpb.GetMembersResponse, error)

GetMembers implements gRPC PDServer.

func (*Server) GetRaftCluster 

func (s *Server) GetRaftCluster() *RaftCluster

GetRaftCluster gets raft cluster. If cluster has not been bootstrapped, return nil.

func (*Server) GetRegion 

func (s *Server) GetRegion(ctx context.Context, request *pdpb.GetRegionRequest) (*pdpb.GetRegionResponse, error)

GetRegion implements gRPC PDServer.

func (*Server) GetRegionByID 

func (s *Server) GetRegionByID(ctx context.Context, request *pdpb.GetRegionByIDRequest) (*pdpb.GetRegionResponse, error)

GetRegionByID implements gRPC PDServer.

func (*Server) GetReplicationConfig 

func (s *Server) GetReplicationConfig() *ReplicationConfig

GetReplicationConfig get the replication config

func (*Server) GetScheduleConfig 

func (s *Server) GetScheduleConfig() *ScheduleConfig

GetScheduleConfig gets the balance config information.

func (*Server) GetStore 

func (s *Server) GetStore(ctx context.Context, request *pdpb.GetStoreRequest) (*pdpb.GetStoreResponse, error)

GetStore implements gRPC PDServer.

func (*Server) ID 

func (s *Server) ID() uint64

ID returns the unique etcd ID for this server in etcd cluster.

func (*Server) IsBootstrapped 

func (s *Server) IsBootstrapped(ctx context.Context, request *pdpb.IsBootstrappedRequest) (*pdpb.IsBootstrappedResponse, error)

IsBootstrapped implements gRPC PDServer.

func (*Server) IsLeader 

func (s *Server) IsLeader() bool

IsLeader returns whether server is leader or not.

func (*Server) Name 

func (s *Server) Name() string

Name returns the unique etcd Name for this server in etcd cluster.

func (*Server) PutClusterConfig 

func (s *Server) PutClusterConfig(ctx context.Context, request *pdpb.PutClusterConfigRequest) (*pdpb.PutClusterConfigResponse, error)

PutClusterConfig implements gRPC PDServer.

func (*Server) PutStore 

func (s *Server) PutStore(ctx context.Context, request *pdpb.PutStoreRequest) (*pdpb.PutStoreResponse, error)

PutStore implements gRPC PDServer.

func (*Server) RegionHeartbeat 

func (s *Server) RegionHeartbeat(ctx context.Context, request *pdpb.RegionHeartbeatRequest) (*pdpb.RegionHeartbeatResponse, error)

RegionHeartbeat implements gRPC PDServer.

func (*Server) ReportSplit 

func (s *Server) ReportSplit(ctx context.Context, request *pdpb.ReportSplitRequest) (*pdpb.ReportSplitResponse, error)

ReportSplit implements gRPC PDServer.

func (*Server) Run 

func (s *Server) Run()

Run runs the pd server.

func (*Server) SetReplicationConfig 

func (s *Server) SetReplicationConfig(cfg ReplicationConfig)

SetReplicationConfig sets the replication config

func (*Server) SetScheduleConfig 

func (s *Server) SetScheduleConfig(cfg ScheduleConfig)

SetScheduleConfig sets the balance config information.

func (*Server) StartEtcd 

func (s *Server) StartEtcd(apiHandler http.Handler) error

StartEtcd starts an embed etcd server with an user handler.

func (*Server) StoreHeartbeat 

func (s *Server) StoreHeartbeat(ctx context.Context, request *pdpb.StoreHeartbeatRequest) (*pdpb.StoreHeartbeatResponse, error)

StoreHeartbeat implements gRPC PDServer.

func (*Server) Tso 

func (s *Server) Tso(stream pdpb.PD_TsoServer) error

Tso implements gRPC PDServer.

type StoreStatus 

type StoreStatus struct {
	*pdpb.StoreStats

	LeaderCount     int
	RegionCount     int
	LastHeartbeatTS time.Time `json:"last_heartbeat_ts"`
	// contains filtered or unexported fields
}

StoreStatus contains information about a store's status.

func (*StoreStatus) GetStartTS 

func (s *StoreStatus) GetStartTS() time.Time

GetStartTS returns the start timestamp.

func (*StoreStatus) GetUptime 

func (s *StoreStatus) GetUptime() time.Duration

GetUptime returns the uptime.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.