remotes

Documentation

Overview

Implements storage.Remote in a way that works with the existing httpserver package.

Index

• type Pool
  • func (p *Pool) AssignRemotes(r int) ([]storage.Remote, error)
  • func (p *Pool) Read(rc storage.ReadConfig) (io.ReadCloser, error)
• type Remote
  • func (r *Remote) Delete(namespace, fn string) error
  • func (r *Remote) HeartBeat(namespace, fn string) (bool, error)
  • func (r *Remote) Initialize(namespace, fn string) error
  • func (r *Remote) Read(rc storage.ReadConfig) (io.ReadCloser, error)
  • func (r *Remote) Replicate(rc storage.RemoteReplicateConfig) (shuttingDown bool, err error)
  • func (r *Remote) String() string

Types

type Pool

type Pool struct {
	// The list of Remotes configured for this cluster.
	Remotes []storage.Remote

	// We also store a mapping of Remotes by the MachineID associated with
	// them. This gives us quick access when performing Read() calls.
	RemotesByMachineID map[uint32]storage.Remote

	// When assigning new Remotes for a primary this will be used to
	// round robin.
	NextRemote     int
	NextRemoteLock sync.Mutex
}

Manages a pool of Remotes that can be used for operations with the storage implementation. The pool manages all the backends and supports the "AssignRemotes" functionality as well which is used when selecting remotes to use for new files.

func (*Pool) AssignRemotes

func (p *Pool) AssignRemotes(r int) ([]storage.Remote, error)

Picks a list of Remotes that should be used for a new master file.

func (*Pool) Read

func (p *Pool) Read(rc storage.ReadConfig) (io.ReadCloser, error)

When a HTTP caller performs a GET against a token it will be processed internally if possible (the file was created on the local machine and is still present, or the replica is hosted on this server) however if the replica was created on a remote machine Blobby will attempt to fetch from that machine since it may still have the file locally. That server will look it up locally and if its not present return an 404 error. However in order to do that properly we need to be able to pass context from the HTTP request that started this operation.

type Remote

type Remote struct {
	// Tracks the name of this Remote. This is used in logging to identify
	// a given remote against others. This may be a nice hostname where the
	// actual configuration points to an IP address directly.
	Name string

	// The URL (Including protocol, port) used to access the blobby
	// server. This is the base so it shouldn't contain any paths at all.
	URL string

	// The numeric ID given to this specific server. This needs to be unique
	// in the cluster.
	ID uint32

	// The http.Client that will be used when making http requests to this
	// specific client. This is useful if you need to support sending TLS
	// requests to a specific IP but still want to perform hostname validation.
	Client *http.Client
}

func (*Remote) Delete

func (r *Remote) Delete(namespace, fn string) error

func (*Remote) HeartBeat

func (r *Remote) HeartBeat(namespace, fn string) (bool, error)

func (*Remote) Initialize

func (r *Remote) Initialize(namespace, fn string) error

func (*Remote) Read

func (r *Remote) Read(rc storage.ReadConfig) (io.ReadCloser, error)

When the storage.Storage object gets a Read() request for a file id that was generated on another machine it will attempt to forward the request to that machine so it can be processed locally on that machine which is cheaper than going to S3.

func (*Remote) Replicate

func (r *Remote) Replicate(
	rc storage.RemoteReplicateConfig,
) (
	shuttingDown bool,
	err error,
)

Replicates data that was written to the primary into the replica. This takes a RemoteReplicateConfig object that contains a bunch of parameters to establish what should be passed to the replica. This returns an error if the replication failed for any reason and a bool that indicates that the replica is going to be shut down soon so further replication should cease.

func (*Remote) String

func (r *Remote) String() string

Returns the name of the remote as a string.