backend

package
v4.0.8+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 11, 2019 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package backend provides storage backend abstraction layer

backend package allows for pluggable back-ends for secrets storage. To implement a new storage back-end you have to supply an object which:

  • implements backend.Backend interface
  • implements backend.NewFunc function

Index

Constants

View Source 
const (
	// DefaultBufferSize is a default circular buffer size
	// used by backends to fan out events
	DefaultBufferSize = 1024
	// DefaultPollStreamPeriod is a default event poll stream period
	DefaultPollStreamPeriod = time.Second
	// DefaultEventsTTL is a default events TTL period
	DefaultEventsTTL = 10 * time.Minute
	// DefaultLargeLimit is used to specify some very large limit when limit is not specified
	// explicitly to prevent OOM
	DefaultLargeLimit = 30000
)
View Source 
const (
	Forever time.Duration = 0
)

Forever means that object TTL will not expire unless deleted

View Source 
const NoLimit = 0

NoLimit specifies no limits

View Source 
const Separator = '/'

Separator is used as a separator between key parts

Variables

This section is empty.

Functions

func AcquireLock 

func AcquireLock(ctx context.Context, backend Backend, lockName string, ttl time.Duration) (err error)

AcquireLock grabs a lock that will be released automatically in TTL

func EarliestExpiry 

func EarliestExpiry(times ...time.Time) time.Time

EarliestExpiry returns first of the otherwise returns empty

func Expiry 

func Expiry(clock clockwork.Clock, ttl time.Duration) time.Time

Expiry converts ttl to expiry time, if ttl is 0 returns empty time

func Key 

func Key(parts ...string) []byte

Key joins parts into path separated by Separator, makes sure path always starts with Separator ("/")

func RangeEnd 

func RangeEnd(key []byte) []byte

RangeEnd returns end of the range for given key

func ReleaseLock 

func ReleaseLock(ctx context.Context, backend Backend, lockName string) error

ReleaseLock forces lock release

func TTL 

func TTL(clock clockwork.Clock, expires time.Time) time.Duration

TTL returns TTL in duration units, rounds up to one second

Types

type Backend 

type Backend interface {
	// Create creates item if it does not exist
	Create(ctx context.Context, i Item) (*Lease, error)

	// Put puts value into backend (creates if it does not
	// exists, updates it otherwise)
	Put(ctx context.Context, i Item) (*Lease, error)

	// CompareAndSwap compares item with existing item
	// and replaces is with replaceWith item
	CompareAndSwap(ctx context.Context, expected Item, replaceWith Item) (*Lease, error)

	// Update updates value in the backend
	Update(ctx context.Context, i Item) (*Lease, error)

	// Get returns a single item or not found error
	Get(ctx context.Context, key []byte) (*Item, error)

	// GetRange returns query range
	GetRange(ctx context.Context, startKey []byte, endKey []byte, limit int) (*GetResult, error)

	// Delete deletes item by key, returns NotFound error
	// if item does not exist
	Delete(ctx context.Context, key []byte) error

	// DeleteRange deletes range of items with keys between startKey and endKey
	DeleteRange(ctx context.Context, startKey, endKey []byte) error

	// KeepAlive keeps object from expiring, updates lease on the existing object,
	// expires contains the new expiry to set on the lease,
	// some backends may ignore expires based on the implementation
	// in case if the lease managed server side
	KeepAlive(ctx context.Context, lease Lease, expires time.Time) error

	// NewWatcher returns a new event watcher
	NewWatcher(ctx context.Context, watch Watch) (Watcher, error)

	// Close closes backend and all associated resources
	Close() error

	// Clock returns clock used by this backend
	Clock() clockwork.Clock

	// CloseWatchers closes all the watchers
	// without closing the backend
	CloseWatchers()
}

Backend implements abstraction over local or remote storage backend

type Batch 

type Batch interface {
	// PutRange puts range of items in one transaction
	PutRange(ctx context.Context, items []Item) error
}

Batch implements some batch methods that are not mandatory for all interfaces, only the ones used in bulk operations.

type BufferWatcher 

type BufferWatcher struct {
	Watch
	// contains filtered or unexported fields
}

BufferWatcher is a watcher connected to the buffer and receiving fan-out events from the watcher

func (*BufferWatcher) Close 

func (w *BufferWatcher) Close() error

Close closes the watcher, could be called multiple times, removes the watcher from the buffer queue

func (*BufferWatcher) Done 

func (w *BufferWatcher) Done() <-chan struct{}

Done channel is closed when watcher is closed

func (*BufferWatcher) Events 

func (w *BufferWatcher) Events() <-chan Event

Events returns events channel

func (*BufferWatcher) String 

func (w *BufferWatcher) String() string

String returns user-friendly representation of the buffer watcher

type CircularBuffer 

type CircularBuffer struct {
	sync.Mutex
	*log.Entry
	// contains filtered or unexported fields
}

CircularBuffer implements in-memory circular buffer of predefined size, that is capable of fan-out of the backend events.

func NewCircularBuffer 

func NewCircularBuffer(ctx context.Context, size int) (*CircularBuffer, error)

NewCircularBuffer returns a new instance of circular buffer

func (*CircularBuffer) Close 

func (c *CircularBuffer) Close() error

Close closes circular buffer and all watchers

func (*CircularBuffer) Events 

func (c *CircularBuffer) Events() []Event

Events returns a copy of records as arranged from start to end

func (*CircularBuffer) NewWatcher 

func (c *CircularBuffer) NewWatcher(ctx context.Context, watch Watch) (Watcher, error)

NewWatcher adds a new watcher to the events buffer

func (*CircularBuffer) Push 

func (c *CircularBuffer) Push(r Event)

Push pushes elements to the queue

func (*CircularBuffer) PushBatch 

func (c *CircularBuffer) PushBatch(events []Event)

PushBatch pushes elements to the queue as a batch

func (*CircularBuffer) Reset 

func (c *CircularBuffer) Reset()

Reset resets all events from the queue and closes all active watchers

func (*CircularBuffer) Size 

func (c *CircularBuffer) Size() int

Size returns circular buffer size

type Config added in v1.2.6 

type Config struct {
	// Type can be "bolt" or "etcd" or "dynamodb"
	Type string `yaml:"type,omitempty"`

	// Params is a generic key/value property bag which allows arbitrary
	// falues to be passed to backend
	Params Params `yaml:",inline"`
}

Config is used for 'storage' config section. It's a combination of values for various backends: 'boltdb', 'etcd', 'filesystem' and 'dynamodb'

type Event 

type Event struct {
	// Type is operation type
	Type OpType
	// Item is event Item
	Item Item
}

Event is a event containing operation with item

type GetResult 

type GetResult struct {
	// Items returns a list of items
	Items []Item
}

GetResult provides the result of GetRange request

type Item 

type Item struct {
	// Key is a key of the key value item
	Key []byte
	// Value is a value of the key value item
	Value []byte
	// Expires is an optional record expiry time
	Expires time.Time
	// ID is a record ID, newer records have newer ids
	ID int64
	// LeaseID is a lease ID, could be set on objects
	// with TTL
	LeaseID int64
}

Item is a key value item

type Items 

type Items []Item

Items is a sortable list of backend items

func (Items) Len 

func (it Items) Len() int

Len is part of sort.Interface.

func (Items) Less 

func (it Items) Less(i, j int) bool

Less is part of sort.Interface.

func (Items) Swap 

func (it Items) Swap(i, j int)

Swap is part of sort.Interface.

type Lease 

type Lease struct {
	// Key is an object representing lease
	Key []byte
	// ID is a lease ID, could be empty
	ID int64
}

Lease represents a lease on the item that can be used to extend item's TTL without updating its contents.

Here is an example of renewing object TTL:

lease, err := backend.Create() lease.Expires = time.Now().Add(time.Second) // Item TTL is extended err = backend.KeepAlive(lease)

func (*Lease) IsEmpty 

func (l *Lease) IsEmpty() bool

IsEmpty returns true if the lease is empty value

type OpType 

type OpType int

OpType specifies operation type 

const (
	// OpInit is returned by the system whenever the system
	// is initialized, init operation is always sent
	// as a first event over the channel, so the client
	// can verify that watch has been established.
	OpInit OpType = iota
	// OpPut is returned for Put events
	OpPut OpType = iota
	// OpDelete is returned for Delete events
	OpDelete OpType = iota
	// OpGet is used for tracking, not present in the event stream
	OpGet OpType = iota
)

func (OpType) String 

func (o OpType) String() string

String returns user-friendly description of the operation

type Params 

type Params map[string]interface{}

Params type defines a flexible unified back-end configuration API. It is just a map of key/value pairs which gets populated by `storage` section in Teleport YAML config.

func (Params) GetString 

func (p Params) GetString(key string) string

GetString returns a string value stored in Params map, or an empty string if nothing is found

type Reporter 

type Reporter struct {
	// ReporterConfig contains reporter wrapper configuration
	ReporterConfig
}

Reporter wraps a Backend implementation and reports statistics about the backend operations

func NewReporter 

func NewReporter(cfg ReporterConfig) (*Reporter, error)

NewReporter returns a new Reporter.

func (*Reporter) Clock 

func (s *Reporter) Clock() clockwork.Clock

Clock returns clock used by this backend

func (*Reporter) Close 

func (s *Reporter) Close() error

Close releases the resources taken up by this backend

func (*Reporter) CloseWatchers 

func (s *Reporter) CloseWatchers()

CloseWatchers closes all the watchers without closing the backend

func (*Reporter) CompareAndSwap 

func (s *Reporter) CompareAndSwap(ctx context.Context, expected Item, replaceWith Item) (*Lease, error)

CompareAndSwap compares item with existing item and replaces is with replaceWith item

func (*Reporter) Create 

func (s *Reporter) Create(ctx context.Context, i Item) (*Lease, error)

Create creates item if it does not exist

func (*Reporter) Delete 

func (s *Reporter) Delete(ctx context.Context, key []byte) error

Delete deletes item by key

func (*Reporter) DeleteRange 

func (s *Reporter) DeleteRange(ctx context.Context, startKey []byte, endKey []byte) error

DeleteRange deletes range of items

func (*Reporter) Get 

func (s *Reporter) Get(ctx context.Context, key []byte) (*Item, error)

Get returns a single item or not found error

func (*Reporter) GetRange 

func (s *Reporter) GetRange(ctx context.Context, startKey []byte, endKey []byte, limit int) (*GetResult, error)

GetRange returns query range

func (*Reporter) KeepAlive 

func (s *Reporter) KeepAlive(ctx context.Context, lease Lease, expires time.Time) error

KeepAlive keeps object from expiring, updates lease on the existing object, expires contains the new expiry to set on the lease, some backends may ignore expires based on the implementation in case if the lease managed server side

func (*Reporter) NewWatcher 

func (s *Reporter) NewWatcher(ctx context.Context, watch Watch) (Watcher, error)

NewWatcher returns a new event watcher

func (*Reporter) Put 

func (s *Reporter) Put(ctx context.Context, i Item) (*Lease, error)

Put puts value into backend (creates if it does not exists, updates it otherwise)

func (*Reporter) Update 

func (s *Reporter) Update(ctx context.Context, i Item) (*Lease, error)

Update updates value in the backend

type ReporterConfig 

type ReporterConfig struct {
	// Backend is a backend to wrap
	Backend Backend
	// TrackTopRequests turns on tracking of top
	// requests on
	TrackTopRequests bool
	// Component is a component name to report
	Component string
}

ReporterConfig configures reporter wrapper

func (*ReporterConfig) CheckAndSetDefaults 

func (r *ReporterConfig) CheckAndSetDefaults() error

CheckAndSetDefaults checks and sets

type ReporterWatcher 

type ReporterWatcher struct {
	Watcher
	Component string
}

ReporterWatcher is a wrapper around backend watcher that reports events

func NewReporterWatcher 

func NewReporterWatcher(ctx context.Context, component string, w Watcher) *ReporterWatcher

NewReporterWatcher creates new reporter watcher instance

type Sanitizer 

type Sanitizer struct {
	// contains filtered or unexported fields
}

Sanitizer wraps a Backend implementation to make sure all values requested of the backend are whitelisted.

func NewSanitizer 

func NewSanitizer(backend Backend) *Sanitizer

NewSanitizer returns a new Sanitizer.

func (*Sanitizer) Backend 

func (s *Sanitizer) Backend() Backend

Backend returns the underlying backend. Useful when knowing the type of backend is important (for example, can the backend support forking).

func (*Sanitizer) Clock 

func (s *Sanitizer) Clock() clockwork.Clock

Clock returns clock used by this backend

func (*Sanitizer) Close 

func (s *Sanitizer) Close() error

Close releases the resources taken up by this backend

func (*Sanitizer) CloseWatchers 

func (s *Sanitizer) CloseWatchers()

CloseWatchers closes all the watchers without closing the backend

func (*Sanitizer) CompareAndSwap 

func (s *Sanitizer) CompareAndSwap(ctx context.Context, expected Item, replaceWith Item) (*Lease, error)

CompareAndSwap compares item with existing item and replaces is with replaceWith item

func (*Sanitizer) Create 

func (s *Sanitizer) Create(ctx context.Context, i Item) (*Lease, error)

Create creates item if it does not exist

func (*Sanitizer) Delete 

func (s *Sanitizer) Delete(ctx context.Context, key []byte) error

Delete deletes item by key

func (*Sanitizer) DeleteRange 

func (s *Sanitizer) DeleteRange(ctx context.Context, startKey []byte, endKey []byte) error

DeleteRange deletes range of items

func (*Sanitizer) Get 

func (s *Sanitizer) Get(ctx context.Context, key []byte) (*Item, error)

Get returns a single item or not found error

func (*Sanitizer) GetRange 

func (s *Sanitizer) GetRange(ctx context.Context, startKey []byte, endKey []byte, limit int) (*GetResult, error)

GetRange returns query range

func (*Sanitizer) KeepAlive 

func (s *Sanitizer) KeepAlive(ctx context.Context, lease Lease, expires time.Time) error

KeepAlive keeps object from expiring, updates lease on the existing object, expires contains the new expiry to set on the lease, some backends may ignore expires based on the implementation in case if the lease managed server side

func (*Sanitizer) NewWatcher 

func (s *Sanitizer) NewWatcher(ctx context.Context, watch Watch) (Watcher, error)

NewWatcher returns a new event watcher

func (*Sanitizer) Put 

func (s *Sanitizer) Put(ctx context.Context, i Item) (*Lease, error)

Put puts value into backend (creates if it does not exists, updates it otherwise)

func (*Sanitizer) Update 

func (s *Sanitizer) Update(ctx context.Context, i Item) (*Lease, error)

Update updates value in the backend

type Watch 

type Watch struct {
	// Name is a watch name set for debugging
	// purposes
	Name string
	// Prefixes specifies prefixes to watch,
	// passed to the backend implementation
	Prefixes [][]byte
	// QueueSize is an optional queue size
	QueueSize int
	// MetricComponent if set will start reporting
	// with a given component metric
	MetricComponent string
}

Watch specifies watcher parameters

func (*Watch) String 

func (w *Watch) String() string

String returns a user-friendly description of the watcher

type Watcher 

type Watcher interface {
	// Events returns channel with events
	Events() <-chan Event

	// Done returns the channel signalling the closure
	Done() <-chan struct{}

	// Close closes the watcher and releases
	// all associated resources
	Close() error
}

Watcher returns watcher

type Wrapper 

type Wrapper struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Wrapper wraps a Backend implementation that can fail on demand.

func NewWrapper 

func NewWrapper(backend Backend) *Wrapper

NewWrapper returns a new Wrapper.

func (*Wrapper) Backend 

func (s *Wrapper) Backend() Backend

Backend returns the underlying backend. Useful when knowing the type of backend is important (for example, can the backend support forking).

func (*Wrapper) Clock 

func (s *Wrapper) Clock() clockwork.Clock

Clock returns clock used by this backend

func (*Wrapper) Close 

func (s *Wrapper) Close() error

Close releases the resources taken up by this backend

func (*Wrapper) CloseWatchers 

func (s *Wrapper) CloseWatchers()

CloseWatchers closes all the watchers without closing the backend

func (*Wrapper) CompareAndSwap 

func (s *Wrapper) CompareAndSwap(ctx context.Context, expected Item, replaceWith Item) (*Lease, error)

CompareAndSwap compares item with existing item and replaces is with replaceWith item

func (*Wrapper) Create 

func (s *Wrapper) Create(ctx context.Context, i Item) (*Lease, error)

Create creates item if it does not exist

func (*Wrapper) Delete 

func (s *Wrapper) Delete(ctx context.Context, key []byte) error

Delete deletes item by key

func (*Wrapper) DeleteRange 

func (s *Wrapper) DeleteRange(ctx context.Context, startKey []byte, endKey []byte) error

DeleteRange deletes range of items

func (*Wrapper) Get 

func (s *Wrapper) Get(ctx context.Context, key []byte) (*Item, error)

Get returns a single item or not found error

func (*Wrapper) GetRange 

func (s *Wrapper) GetRange(ctx context.Context, startKey []byte, endKey []byte, limit int) (*GetResult, error)

GetRange returns query range

func (*Wrapper) GetReadError 

func (s *Wrapper) GetReadError() error

GetReadError returns error to be returned by read backend operations

func (*Wrapper) KeepAlive 

func (s *Wrapper) KeepAlive(ctx context.Context, lease Lease, expires time.Time) error

KeepAlive keeps object from expiring, updates lease on the existing object, expires contains the new expiry to set on the lease, some backends may ignore expires based on the implementation in case if the lease managed server side

func (*Wrapper) NewWatcher 

func (s *Wrapper) NewWatcher(ctx context.Context, watch Watch) (Watcher, error)

NewWatcher returns a new event watcher

func (*Wrapper) Put 

func (s *Wrapper) Put(ctx context.Context, i Item) (*Lease, error)

Put puts value into backend (creates if it does not exists, updates it otherwise)

func (*Wrapper) SetReadError 

func (s *Wrapper) SetReadError(err error)

SetReadError sets error to be returned by read backend operations

func (*Wrapper) Update 

func (s *Wrapper) Update(ctx context.Context, i Item) (*Lease, error)

Update updates value in the backend

Source Files

View all Source files

Directories

Path Synopsis
Package dynamodbDynamoDBBackend implements DynamoDB storage backend for Teleport auth service, similar to etcd backend.
 Package dynamodbDynamoDBBackend implements DynamoDB storage backend for Teleport auth service, similar to etcd backend.
Package etcdbk implements Etcd powered backend
 Package etcdbk implements Etcd powered backend
Package legacy stores previous, unsupported versions of the backend and is used for migrations
 Package legacy stores previous, unsupported versions of the backend and is used for migrations
boltbk
Package boltbk implements BoltDB backed backend for standalone instances This is a legacy backend which only exists for backward compatibility purposes Production Teleport clusters should be using either etcd or DynamoDB backends.
 Package boltbk implements BoltDB backed backend for standalone instances This is a legacy backend which only exists for backward compatibility purposes Production Teleport clusters should be using either etcd or DynamoDB backends.
dir
dir package implements backend.Backend interface using the filesystem.
 dir package implements backend.Backend interface using the filesystem.
test
Package test contains a backend acceptance test suite that is backend implementation independent each backend will use the suite to test itself
 Package test contains a backend acceptance test suite that is backend implementation independent each backend will use the suite to test itself
Package lite implements SQLite backend used for local persistent caches in proxies and nodes and for standalone auth service deployments.
 Package lite implements SQLite backend used for local persistent caches in proxies and nodes and for standalone auth service deployments.
Package memory implements backend interface using a combination of Minheap (to store expiring items) and B-Tree for storing sorted dictionary of items.
 Package memory implements backend interface using a combination of Minheap (to store expiring items) and B-Tree for storing sorted dictionary of items.
Package test contains a backend acceptance test suite that is backend implementation independent each backend will use the suite to test itself
 Package test contains a backend acceptance test suite that is backend implementation independent each backend will use the suite to test itself

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.