physical

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 25, 2016 License: MPL-2.0 Imports: 39 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// DefaultDynamoDBRegion is used when no region is configured
	// explicitely.
	DefaultDynamoDBRegion = "us-east-1"
	// DefaultDynamoDBTableName is used when no table name
	// is configured explicitely.
	DefaultDynamoDBTableName = "vault-dynamodb-backend"

	// DefaultDynamoDBReadCapacity is the default read capacity
	// that is used when none is configured explicitely.
	DefaultDynamoDBReadCapacity = 5
	// DefaultDynamoDBWriteCapacity is the default write capacity
	// that is used when none is configured explicitely.
	DefaultDynamoDBWriteCapacity = 5

	// DynamoDBEmptyPath is the string that is used instead of
	// empty strings when stored in DynamoDB.
	DynamoDBEmptyPath = " "
	// DynamoDBLockPrefix is the prefix used to mark DynamoDB records
	// as locks. This prefix causes them not to be returned by
	// List operations.
	DynamoDBLockPrefix = "_"

	// DynamoDBLockRetryInterval is the amount of time to wait
	// if a lock fails before trying again.
	DynamoDBLockRetryInterval = time.Second
	// DynamoDBWatchRetryMax is the number of times to re-try a
	// failed watch before signaling that leadership is lost.
	DynamoDBWatchRetryMax = 5
	// DynamoDBWatchRetryInterval is the amount of time to wait
	// if a watch fails before trying again.
	DynamoDBWatchRetryInterval = 5 * time.Second
)
View Source 
const (
	// Ideally, this prefix would match the "_" used in the file backend, but
	// that prefix has special meaining in etcd. Specifically, it excludes those
	// entries from directory listings.
	EtcdNodeFilePrefix = "."

	// The lock prefix can (and probably should) cause an entry to be excluded
	// from diretory listings, so "_" works here.
	EtcdNodeLockPrefix = "_"

	// The delimiter is the same as the `-C` flag of etcdctl.
	EtcdMachineDelimiter = ","

	// The lock TTL matches the default that Consul API uses, 15 seconds.
	EtcdLockTTL = 15 * time.Second

	// The amount of time to wait between the semaphore key renewals
	EtcdLockRenewInterval = 5 * time.Second

	// The ammount of time to wait if a watch fails before trying again.
	EtcdWatchRetryInterval = time.Second

	// The number of times to re-try a failed watch before signaling that leadership is lost.
	EtcdWatchRetryMax = 5
)
View Source 
const (
	// DefaultCacheSize is used if no cache size is specified for NewCache
	DefaultCacheSize = 32 * 1024
)
View Source 
const DefaultParallelOperations = 128
View Source 
const (
	// ZKNodeFilePrefix is prefixed to any "files" in ZooKeeper,
	// so that they do not collide with directory entries. Otherwise,
	// we cannot delete a file if the path is a full-prefix of another
	// key.
	ZKNodeFilePrefix = "_"
)

Variables

View Source 
var (
	EtcdSyncConfigError          = errors.New("client setup failed: unable to parse etcd sync field in config")
	EtcdSyncClusterError         = errors.New("client setup failed: unable to sync etcd cluster")
	EtcdAddressError             = errors.New("client setup failed: address must be valid URL (ex. 'scheme://host:port')")
	EtcdSemaphoreKeysEmptyError  = errors.New("lock queue is empty")
	EtcdLockHeldError            = errors.New("lock already held")
	EtcdLockNotHeldError         = errors.New("lock not held")
	EtcdSemaphoreKeyRemovedError = errors.New("semaphore key removed before lock aquisition")
)
View Source 
var BuiltinBackends = map[string]Factory{
	"inmem": func(map[string]string) (Backend, error) {
		return NewInmem(), nil
	},
	"consul":     newConsulBackend,
	"zookeeper":  newZookeeperBackend,
	"file":       newFileBackend,
	"s3":         newS3Backend,
	"dynamodb":   newDynamoDBBackend,
	"etcd":       newEtcdBackend,
	"mysql":      newMySQLBackend,
	"postgresql": newPostgreSQLBackend,
}

BuiltinBackends is the list of built-in physical backends that can be used with NewBackend.

Functions

This section is empty.

Types

type AdvertiseDetect added in v0.1.2 

type AdvertiseDetect interface {
	// DetectHostAddr is used to detect the host address
	DetectHostAddr() (string, error)
}

AdvertiseDetect is an optional interface that an HABackend can implement. If they do, an advertise address can be automatically detected.

type Backend 

type Backend interface {
	// Put is used to insert or update an entry
	Put(entry *Entry) error

	// Get is used to fetch an entry
	Get(key string) (*Entry, error)

	// Delete is used to permanently delete an entry
	Delete(key string) error

	// List is used ot list all the keys under a given
	// prefix, up to the next prefix.
	List(prefix string) ([]string, error)
}

Backend is the interface required for a physical backend. A physical backend is used to durably store data outside of Vault. As such, it is completely untrusted, and is only accessed via a security barrier. The backends must represent keys in a hierarchical manner. All methods are expected to be thread safe.

func NewBackend 

func NewBackend(t string, conf map[string]string) (Backend, error)

NewBackend returns a new backend with the given type and configuration. The backend is looked up in the BuiltinBackends variable.

type Cache 

type Cache struct {
	// contains filtered or unexported fields
}

Cache is used to wrap an underlying physical backend and provide an LRU cache layer on top. Most of the reads done by Vault are for policy objects so there is a large read reduction by using a simple write-through cache.

func NewCache 

func NewCache(b Backend, size int) *Cache

NewCache returns a physical cache of the given size. If no size is provided, the default size is used.

func (*Cache) Delete 

func (c *Cache) Delete(key string) error

func (*Cache) Get 

func (c *Cache) Get(key string) (*Entry, error)

func (*Cache) List 

func (c *Cache) List(prefix string) ([]string, error)

func (*Cache) Purge 

func (c *Cache) Purge()

Purge is used to clear the cache

func (*Cache) Put 

func (c *Cache) Put(entry *Entry) error

type ConsulBackend 

type ConsulBackend struct {
	// contains filtered or unexported fields
}

ConsulBackend is a physical backend that stores data at specific prefix within Consul. It is used for most production situations as it allows Vault to run on multiple machines in a highly-available manner.

func (*ConsulBackend) Delete 

func (c *ConsulBackend) Delete(key string) error

Delete is used to permanently delete an entry

func (*ConsulBackend) DetectHostAddr added in v0.1.2 

func (c *ConsulBackend) DetectHostAddr() (string, error)

DetectHostAddr is used to detect the host address by asking the Consul agent

func (*ConsulBackend) Get 

func (c *ConsulBackend) Get(key string) (*Entry, error)

Get is used to fetch an entry

func (*ConsulBackend) List 

func (c *ConsulBackend) List(prefix string) ([]string, error)

List is used to list all the keys under a given prefix, up to the next prefix.

func (*ConsulBackend) LockWith 

func (c *ConsulBackend) LockWith(key, value string) (Lock, error)

Lock is used for mutual exclusion based on the given key.

func (*ConsulBackend) Put 

func (c *ConsulBackend) Put(entry *Entry) error

Put is used to insert or update an entry

type ConsulLock 

type ConsulLock struct {
	// contains filtered or unexported fields
}

ConsulLock is used to provide the Lock interface backed by Consul

func (*ConsulLock) Lock 

func (c *ConsulLock) Lock(stopCh <-chan struct{}) (<-chan struct{}, error)

func (*ConsulLock) Unlock 

func (c *ConsulLock) Unlock() error

func (*ConsulLock) Value 

func (c *ConsulLock) Value() (bool, string, error)

type DynamoDBBackend added in v0.5.0 

type DynamoDBBackend struct {
	// contains filtered or unexported fields
}

DynamoDBBackend is a physical backend that stores data in a DynamoDB table. It can be run in high-availability mode as DynamoDB has locking capabilities.

func (*DynamoDBBackend) Delete added in v0.5.0 

func (d *DynamoDBBackend) Delete(key string) error

Delete is used to permanently delete an entry

func (*DynamoDBBackend) Get added in v0.5.0 

func (d *DynamoDBBackend) Get(key string) (*Entry, error)

Get is used to fetch an entry

func (*DynamoDBBackend) List added in v0.5.0 

func (d *DynamoDBBackend) List(prefix string) ([]string, error)

List is used to list all the keys under a given prefix, up to the next prefix.

func (*DynamoDBBackend) LockWith added in v0.5.0 

func (d *DynamoDBBackend) LockWith(key, value string) (Lock, error)

LockWith is used for mutual exclusion based on the given key.

func (*DynamoDBBackend) Put added in v0.5.0 

func (d *DynamoDBBackend) Put(entry *Entry) error

Put is used to insert or update an entry

type DynamoDBLock added in v0.5.0 

type DynamoDBLock struct {
	// contains filtered or unexported fields
}

DynamoDBLock implements a lock using an DynamoDB client.

func (*DynamoDBLock) Lock added in v0.5.0 

func (l *DynamoDBLock) Lock(stopCh <-chan struct{}) (doneCh <-chan struct{}, retErr error)

Lock tries to acquire the lock by repeatedly trying to create a record in the DynamoDB table. It will block until either the stop channel is closed or the lock could be acquired successfully. The returned channel will be closed once the lock is deleted or changed in the DynamoDB table.

func (*DynamoDBLock) Unlock added in v0.5.0 

func (l *DynamoDBLock) Unlock() error

Unlock releases the lock by deleting the lock record from the DynamoDB table.

func (*DynamoDBLock) Value added in v0.5.0 

func (l *DynamoDBLock) Value() (bool, string, error)

Value checks whether or not the lock is held by any instance of DynamoDBLock, including this one, and returns the current value.

type DynamoDBRecord added in v0.5.0 

type DynamoDBRecord struct {
	Path  string
	Key   string
	Value []byte
}

DynamoDBRecord is the representation of a vault entry in DynamoDB. The vault key is split up into two components (Path and Key) in order to allow more efficient listings.

type Entry 

type Entry struct {
	Key   string
	Value []byte
}

Entry is used to represent data stored by the physical backend

type EtcdBackend added in v0.2.0 

type EtcdBackend struct {
	// contains filtered or unexported fields
}

EtcdBackend is a physical backend that stores data at specific prefix within Etcd. It is used for most production situations as it allows Vault to run on multiple machines in a highly-available manner.

func (*EtcdBackend) Delete added in v0.2.0 

func (c *EtcdBackend) Delete(key string) error

Delete is used to permanently delete an entry.

func (*EtcdBackend) Get added in v0.2.0 

func (c *EtcdBackend) Get(key string) (*Entry, error)

Get is used to fetch an entry.

func (*EtcdBackend) List added in v0.2.0 

func (c *EtcdBackend) List(prefix string) ([]string, error)

List is used to list all the keys under a given prefix, up to the next prefix.

func (*EtcdBackend) LockWith added in v0.2.0 

func (c *EtcdBackend) LockWith(key, value string) (Lock, error)

Lock is used for mutual exclusion based on the given key.

func (*EtcdBackend) Put added in v0.2.0 

func (c *EtcdBackend) Put(entry *Entry) error

Put is used to insert or update an entry.

type EtcdLock added in v0.2.0 

type EtcdLock struct {
	// contains filtered or unexported fields
}

EtcdLock emplements a lock using and etcd backend.

func (*EtcdLock) Lock added in v0.2.0 

func (c *EtcdLock) Lock(stopCh <-chan struct{}) (doneCh <-chan struct{}, retErr error)

Lock attempts to aquire the lock by waiting for a new semaphore key in etcd to become the first in the queue and will block until it is successful or it recieves a signal on the provided channel. The returned channel will be closed when the lock is lost, either by an explicit call to Unlock or by the associated semaphore key in etcd otherwise being deleted or expiring.

If the lock is currently held by this instance of EtcdLock, Lock will return an EtcdLockHeldError error.

func (*EtcdLock) Unlock added in v0.2.0 

func (c *EtcdLock) Unlock() error

Unlock releases the lock by deleting the associated semaphore key in etcd.

If the lock is not currently held by this instance of EtcdLock, Unlock will return an EtcdLockNotHeldError error.

func (*EtcdLock) Value added in v0.2.0 

func (c *EtcdLock) Value() (bool, string, error)

Value checks whether or not the lock is held by any instance of EtcdLock, including this one, and returns the current value.

type Factory 

type Factory func(map[string]string) (Backend, error)

Factory is the factory function to create a physical backend.

type FileBackend 

type FileBackend struct {
	Path string
	// contains filtered or unexported fields
}

FileBackend is a physical backend that stores data on disk at a given file path. It can be used for durable single server situations, or to develop locally where durability is not critical.

WARNING: the file backend implementation is currently extremely unsafe and non-performant. It is meant mostly for local testing and development. It can be improved in the future.

func (*FileBackend) Delete 

func (b *FileBackend) Delete(k string) error

func (*FileBackend) Get 

func (b *FileBackend) Get(k string) (*Entry, error)

func (*FileBackend) List 

func (b *FileBackend) List(prefix string) ([]string, error)

func (*FileBackend) Put 

func (b *FileBackend) Put(entry *Entry) error

type HABackend 

type HABackend interface {
	// LockWith is used for mutual exclusion based on the given key.
	LockWith(key, value string) (Lock, error)
}

HABackend is an extentions to the standard physical backend to support high-availability. Vault only expects to use mutual exclusion to allow multiple instances to act as a hot standby for a leader that services all requests.

type InmemBackend 

type InmemBackend struct {
	// contains filtered or unexported fields
}

InmemBackend is an in-memory only physical backend. It is useful for testing and development situations where the data is not expected to be durable.

func NewInmem 

func NewInmem() *InmemBackend

NewInmem constructs a new in-memory backend

func (*InmemBackend) Delete 

func (i *InmemBackend) Delete(key string) error

Delete is used to permanently delete an entry

func (*InmemBackend) Get 

func (i *InmemBackend) Get(key string) (*Entry, error)

Get is used to fetch an entry

func (*InmemBackend) List 

func (i *InmemBackend) List(prefix string) ([]string, error)

List is used ot list all the keys under a given prefix, up to the next prefix.

func (*InmemBackend) Put 

func (i *InmemBackend) Put(entry *Entry) error

Put is used to insert or update an entry

type InmemHABackend 

type InmemHABackend struct {
	InmemBackend
	// contains filtered or unexported fields
}

func NewInmemHA 

func NewInmemHA() *InmemHABackend

NewInmemHA constructs a new in-memory HA backend. This is only for testing.

func (*InmemHABackend) LockMapSize added in v0.5.0 

func (i *InmemHABackend) LockMapSize() int

LockMapSize is used in some tests to determine whether this backend has ever been used for HA purposes rather than simply for storage

func (*InmemHABackend) LockWith 

func (i *InmemHABackend) LockWith(key, value string) (Lock, error)

LockWith is used for mutual exclusion based on the given key.

type InmemLock 

type InmemLock struct {
	// contains filtered or unexported fields
}

InmemLock is an in-memory Lock implementation for the HABackend

func (*InmemLock) Lock 

func (i *InmemLock) Lock(stopCh <-chan struct{}) (<-chan struct{}, error)

func (*InmemLock) Unlock 

func (i *InmemLock) Unlock() error

func (*InmemLock) Value 

func (i *InmemLock) Value() (bool, string, error)

type Lock 

type Lock interface {
	// Lock is used to acquire the given lock
	// The stopCh is optional and if closed should interrupt the lock
	// acquisition attempt. The return struct should be closed when
	// leadership is lost.
	Lock(stopCh <-chan struct{}) (<-chan struct{}, error)

	// Unlock is used to release the lock
	Unlock() error

	// Returns the value of the lock and if it is held
	Value() (bool, string, error)
}

type MySQLBackend added in v0.2.0 

type MySQLBackend struct {
	// contains filtered or unexported fields
}

MySQLBackend is a physical backend that stores data within MySQL database.

func (*MySQLBackend) Delete added in v0.2.0 

func (m *MySQLBackend) Delete(key string) error

Delete is used to permanently delete an entry

func (*MySQLBackend) Get added in v0.2.0 

func (m *MySQLBackend) Get(key string) (*Entry, error)

Get is used to fetch and entry.

func (*MySQLBackend) List added in v0.2.0 

func (m *MySQLBackend) List(prefix string) ([]string, error)

List is used to list all the keys under a given prefix, up to the next prefix.

func (*MySQLBackend) Put added in v0.2.0 

func (m *MySQLBackend) Put(entry *Entry) error

Put is used to insert or update an entry.

type PermitPool added in v0.4.0 

type PermitPool struct {
	// contains filtered or unexported fields
}

PermitPool is a wrapper around a semaphore library to keep things agnostic

func NewPermitPool added in v0.4.0 

func NewPermitPool(permits int) *PermitPool

NewPermitPool returns a new permit pool with the provided number of permits

func (*PermitPool) Acquire added in v0.4.0 

func (c *PermitPool) Acquire()

Acquire returns when a permit has been acquired

func (*PermitPool) Release added in v0.4.0 

func (c *PermitPool) Release()

Release returns a permit to the pool

type PostgreSQLBackend added in v0.5.0 

type PostgreSQLBackend struct {
	// contains filtered or unexported fields
}

PostgreSQL Backend is a physical backend that stores data within a PostgreSQL database.

func (*PostgreSQLBackend) Delete added in v0.5.0 

func (m *PostgreSQLBackend) Delete(fullPath string) error

Delete is used to permanently delete an entry

func (*PostgreSQLBackend) Get added in v0.5.0 

func (m *PostgreSQLBackend) Get(fullPath string) (*Entry, error)

Get is used to fetch and entry.

func (*PostgreSQLBackend) List added in v0.5.0 

func (m *PostgreSQLBackend) List(prefix string) ([]string, error)

List is used to list all the keys under a given prefix, up to the next prefix.

func (*PostgreSQLBackend) Put added in v0.5.0 

func (m *PostgreSQLBackend) Put(entry *Entry) error

Put is used to insert or update an entry.

type S3Backend added in v0.2.0 

type S3Backend struct {
	// contains filtered or unexported fields
}

S3Backend is a physical backend that stores data within an S3 bucket.

func (*S3Backend) Delete added in v0.2.0 

func (s *S3Backend) Delete(key string) error

Delete is used to permanently delete an entry

func (*S3Backend) Get added in v0.2.0 

func (s *S3Backend) Get(key string) (*Entry, error)

Get is used to fetch an entry

func (*S3Backend) List added in v0.2.0 

func (s *S3Backend) List(prefix string) ([]string, error)

List is used to list all the keys under a given prefix, up to the next prefix.

func (*S3Backend) Put added in v0.2.0 

func (s *S3Backend) Put(entry *Entry) error

Put is used to insert or update an entry

type ZookeeperBackend added in v0.1.2 

type ZookeeperBackend struct {
	// contains filtered or unexported fields
}

ZookeeperBackend is a physical backend that stores data at specific prefix within Zookeeper. It is used in production situations as it allows Vault to run on multiple machines in a highly-available manner.

func (*ZookeeperBackend) Delete added in v0.1.2 

func (c *ZookeeperBackend) Delete(key string) error

Delete is used to permanently delete an entry

func (*ZookeeperBackend) Get added in v0.1.2 

func (c *ZookeeperBackend) Get(key string) (*Entry, error)

Get is used to fetch an entry

func (*ZookeeperBackend) List added in v0.1.2 

func (c *ZookeeperBackend) List(prefix string) ([]string, error)

List is used ot list all the keys under a given prefix, up to the next prefix.

func (*ZookeeperBackend) LockWith added in v0.2.0 

func (c *ZookeeperBackend) LockWith(key, value string) (Lock, error)

LockWith is used for mutual exclusion based on the given key.

func (*ZookeeperBackend) Put added in v0.1.2 

func (c *ZookeeperBackend) Put(entry *Entry) error

Put is used to insert or update an entry

type ZookeeperHALock added in v0.2.0 

type ZookeeperHALock struct {
	// contains filtered or unexported fields
}

ZookeeperHALock is a Zookeeper Lock implementation for the HABackend

func (*ZookeeperHALock) Lock added in v0.2.0 

func (i *ZookeeperHALock) Lock(stopCh <-chan struct{}) (<-chan struct{}, error)

func (*ZookeeperHALock) Unlock added in v0.2.0 

func (i *ZookeeperHALock) Unlock() error

func (*ZookeeperHALock) Value added in v0.2.0 

func (i *ZookeeperHALock) Value() (bool, string, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.