storehost

package
v0.0.0-...-f9f87ea Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 1, 2017 License: MIT Imports: 30 Imported by: 0

Documentation

Index

Constants

View Source 
const (

	// JobTypeReReplication is the job type for re-replication. The source of re-replication is another store
	JobTypeReReplication

	// JobTypeRemoteReplication is the job type for remote replication. The source of remote replication is
	// replicator(which will receive message from remote replicator)
	JobTypeRemoteReplication
)
View Source 
const (
	// SMReadWrite allows both read and write
	SMReadWrite = iota
	// SMReadOnly allows read only
	SMReadOnly
)

Variables

View Source 
var ReportInterval = time.Duration(1 * time.Minute)

ReportInterval determines how often reporting housekeeping will happen. It is exported to allow test to modify this value

View Source 
var ReportPause *int32

ReportPause is used for testing to allow the reporter to be paused at will

Functions

func QueueMonitorPause 

func QueueMonitorPause()

QueueMonitorPause is for testing use. It causes the queueMonitor to wait at the next tick, instead of doing its normal processing

func QueueMonitorUnpause 

func QueueMonitorUnpause()

QueueMonitorUnpause is for testing use. It causes the queueMonitor to operate normally.

Types

type CreditLine 

type CreditLine interface {
	Borrow() (credits int32)
	Return(credits int32)
	Close()
}

CreditLine defines the credit-line interface

type CreditMgr 

type CreditMgr interface {
	NewCreditLine() CreditLine
	Close() // close credit mgr
}

CreditMgr defines the credit-manager interface

func NewCreditMgr 

func NewCreditMgr(totalCredits int32) CreditMgr

NewCreditMgr returns an instance of CreditMgr

type ExtentManager 

type ExtentManager struct {

	// lock, primarily used to synchronize the 'extents' map (below)
	sync.RWMutex
	// contains filtered or unexported fields
}

ExtentManager contains the map of all open extent-contexts

func NewExtentManager 

func NewExtentManager(storeMgr storage.StoreManager, m3Client metrics.Client, hostMetrics *load.HostMetrics, log bark.Logger) (xMgr *ExtentManager)

NewExtentManager initializes and returns a new 'extent manager'

func (*ExtentManager) IsExtentOpenedForReplication 

func (xMgr *ExtentManager) IsExtentOpenedForReplication(extentID string) bool

IsExtentOpenedForReplication checks whether an extent is already opened for replication

func (*ExtentManager) OpenExtent 

func (xMgr *ExtentManager) OpenExtent(id uuid.UUID, mode Mode, intent OpenIntent) (x *ExtentObj, err error)

OpenExtent opens DB and initializes state for a given extent and returns a 'handle' to it

type ExtentObj 

type ExtentObj struct {
	// contains filtered or unexported fields
}

ExtentObj is allocated one per "open" to an extent

func (*ExtentObj) Close 

func (x *ExtentObj) Close()

Close closes this handle to the extent

func (*ExtentObj) Delete 

func (x *ExtentObj) Delete()

Delete marks the extent to be deleted when the last reference goes away

func (*ExtentObj) Mode 

func (x *ExtentObj) Mode() Mode

Mode returns this extent's mode (ie, AppendOnly or TimerQueue)

type Mode 

type Mode int

Mode of operation: timer-queue or append-only mode 

const (

	// AppendOnly mode (assumes all extents are in "append only" message queues)
	AppendOnly Mode

	// TimerQueue mode (assumes all extents are in "timer queues")
	TimerQueue

	// Log mode (similar to AppendOnly which is gated on watermark)
	Log
)

func (Mode) String 

func (t Mode) String() string

type OpenIntent 

type OpenIntent int

OpenIntent is used to indicate what the particular call to OpenExtent is intended for 

const (

	// OpenIntentAppendStream is used by OpenAppendStream
	OpenIntentAppendStream OpenIntent

	// OpenIntentReadStream is used by OpenReadStream
	OpenIntentReadStream

	// OpenIntentSealExtent is used by SealExtent
	OpenIntentSealExtent

	// OpenIntentGetAddressFromTimestamp is used by GetAddressFromTimestamp
	OpenIntentGetAddressFromTimestamp

	// OpenIntentGetExtentInfo is used by GetExtentInfo
	OpenIntentGetExtentInfo

	// OpenIntentPurgeMessages is used by PurgeMessages
	OpenIntentPurgeMessages

	// OpenIntentReplicateExtent is used by ReplicateExtent
	OpenIntentReplicateExtent
)

func (OpenIntent) String 

func (t OpenIntent) String() string

type Options 

type Options struct {
	Store   Store
	BaseDir string
}

Options are the arguments passed to the storehost

type QueueMonitor 

type QueueMonitor interface {
	common.Daemon
}

QueueMonitor keeps record of extents progress, and flush to metastore periodically

type ReplicationJob 

type ReplicationJob struct {
	// contains filtered or unexported fields
}

ReplicationJob holds the context for a replication job

func (*ReplicationJob) Done 

func (t *ReplicationJob) Done() <-chan struct{}

Done returns a channel that is signaled when the replication job completes

func (*ReplicationJob) Error 

func (t *ReplicationJob) Error() error

Error returns the error from the replication job

func (*ReplicationJob) Start 

func (t *ReplicationJob) Start() error

Start starts the replication job

func (*ReplicationJob) Stop 

func (t *ReplicationJob) Stop()

Stop stops the replication job

type ReplicationJobRunner 

type ReplicationJobRunner interface {
	common.Daemon
}

ReplicationJobRunner periodically queries metadata and start replication job for extents if there's no existing job running The runner is needed to handle error cases like store restart, etc Eventually it should be replaced with a more general purpose job framework

func NewReplicationJobRunner 

func NewReplicationJobRunner(mClient metadata.TChanMetadataService, store *StoreHost, logger bark.Logger) ReplicationJobRunner

NewReplicationJobRunner returns an instance of ReplicationJobRunner

type ReplicationJobType 

type ReplicationJobType int

ReplicationJobType is the job type

type ReplicationManager 

type ReplicationManager struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

ReplicationManager holds the context for replication manager

func NewReplicationManager 

func NewReplicationManager(
	xMgr *ExtentManager,
	m3Client metrics.Client,
	mClient metadata.TChanMetadataService,
	log bark.Logger,
	hostID string,
	wsConn common.WSConnector,
) *ReplicationManager

NewReplicationManager initializes and returns a new 'replication manager'

func (*ReplicationManager) NewJob 

func (t *ReplicationManager) NewJob(jobType ReplicationJobType, args *replicationArgs, log bark.Logger) *ReplicationJob

NewJob schedules starts a new replication job

type StorageMode 

type StorageMode int32

StorageMode defines the read write mode of the storage host

type StorageMonitor 

type StorageMonitor interface {
	common.Daemon
	GetStorageMode() StorageMode
}

StorageMonitor keep monitoring disk usage, and log/alert/trigger necessary handling

func NewStorageMonitor 

func NewStorageMonitor(store *StoreHost, m3Client metrics.Client, hostMetrics *load.HostMetrics, logger bark.Logger, path string) StorageMonitor

NewStorageMonitor returns an instance of NewStorageMonitor.

type StorageStatus 

type StorageStatus int32

StorageStatus defines the different storage status

type Store 

type Store int

Store indicates the underlying storage to use 

const (

	// Rockstor for store
	Rockstor Store

	// Chunky for store
	Chunky

	// ManyRocks for store
	ManyRocks

	// RockCFstor for store
	RockCFstor
)

func (Store) String 

func (t Store) String() string

type StoreHost 

type StoreHost struct {
	common.SCommon
	// contains filtered or unexported fields
}

StoreHost is the main server class for StoreHosts

func NewStoreHost 

func NewStoreHost(serviceName string, sCommon common.SCommon, mClient metadata.TChanMetadataService, opts *Options) (*StoreHost, []thrift.TChanServer)

NewStoreHost is the constructor for store host

func (*StoreHost) DisableWrite 

func (t *StoreHost) DisableWrite()

DisableWrite disables all the write

func (*StoreHost) EnableWrite 

func (t *StoreHost) EnableWrite()

EnableWrite enables write mode

func (*StoreHost) GetAddressFromTimestamp 

func (t *StoreHost) GetAddressFromTimestamp(ctx thrift.Context, req *store.GetAddressFromTimestampRequest) (res *store.GetAddressFromTimestampResult_, err error)

GetAddressFromTimestamp is the implementation of the thrift handler for store host

func (*StoreHost) GetExtentInfo 

func (t *StoreHost) GetExtentInfo(ctx thrift.Context, extReq *store.GetExtentInfoRequest) (*store.ExtentInfo, error)

GetExtentInfo is the implementation of the thrift handler for the store host

func (*StoreHost) NewQueueMonitor 

func (host *StoreHost) NewQueueMonitor(mClient metadata.TChanMetadataService, store *StoreHost, logger bark.Logger) QueueMonitor

NewQueueMonitor returns an instance of NewQueueMonitor that can be used to monitor and report all the extents progress on a store host.

func (*StoreHost) OpenAppendStream 

func (t *StoreHost) OpenAppendStream(ctx thrift.Context, call storeStream.BStoreOpenAppendStreamInCall) error

OpenAppendStream is the implementation of the thrift handler for the store host

func (*StoreHost) OpenAppendStreamHandler 

func (t *StoreHost) OpenAppendStreamHandler(w http.ResponseWriter, r *http.Request)

OpenAppendStreamHandler is websocket handler for opening write stream

func (*StoreHost) OpenReadStream 

func (t *StoreHost) OpenReadStream(ctx thrift.Context, call storeStream.BStoreOpenReadStreamInCall) error

OpenReadStream is the implementation of the thrift handler for the store host

func (*StoreHost) OpenReadStreamHandler 

func (t *StoreHost) OpenReadStreamHandler(w http.ResponseWriter, r *http.Request)

OpenReadStreamHandler is websocket handler for opening read stream

func (*StoreHost) PurgeMessages 

func (t *StoreHost) PurgeMessages(ctx thrift.Context, req *store.PurgeMessagesRequest) (res *store.PurgeMessagesResult_, err error)

PurgeMessages is the implementation of the thrift handler for the store host

func (*StoreHost) ReadMessages 

func (t *StoreHost) ReadMessages(
	ctx thrift.Context, req *store.ReadMessagesRequest) (result *store.ReadMessagesResult_, err error)

ReadMessages reads a set of messages start from the set StartAddress

func (*StoreHost) RegisterWSHandler 

func (t *StoreHost) RegisterWSHandler() *http.ServeMux

RegisterWSHandler is the implementation of WSService interface

func (*StoreHost) RemoteReplicateExtent 

func (t *StoreHost) RemoteReplicateExtent(tCtx thrift.Context, req *store.RemoteReplicateExtentRequest) (err error)

RemoteReplicateExtent replicates a remote extent from replicator

func (*StoreHost) ReplicateExtent 

func (t *StoreHost) ReplicateExtent(tCtx thrift.Context, req *store.ReplicateExtentRequest) (err error)

ReplicateExtent creates a replica of the extent in the local store

func (*StoreHost) Report 

func (t *StoreHost) Report(reporter common.LoadReporter)

Report is used for reporting Host specific load to controller

func (*StoreHost) SealExtent 

func (t *StoreHost) SealExtent(ctx thrift.Context, req *store.SealExtentRequest) (err error)

SealExtent is the implementation of the thrift handler for the store host

func (*StoreHost) Shutdown 

func (t *StoreHost) Shutdown()

Shutdown storehost

func (*StoreHost) Start 

func (t *StoreHost) Start(thriftService []thrift.TChanServer)

Start starts the storehost service

func (*StoreHost) Stop 

func (t *StoreHost) Stop()

Stop stops the service

type Throttler 

type Throttler struct {
	// contains filtered or unexported fields
}

Throttler holds the context for a throttler instance.

func NewThrottler 

func NewThrottler(refillTokens uint64, refillInterval time.Duration) *Throttler

NewThrottler instantiates a new Throttler object - refillTokens: number of tokens to issue every refillInterval (below) - refillInterval: interval at which 'refillTokens' should be issued

func (*Throttler) Allow 

func (t *Throttler) Allow() bool

Allow checks to see if a particular request should be allowed; ie, it would returns true if we are under the throttling threshold, otherwise false.

func (*Throttler) AllowN 

func (t *Throttler) AllowN(n uint64) bool

AllowN checks to see if we will be allowed to do a batch of 'n' tokens; it would return true if we are under the throttling threshold, otherwise false.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.