server

Documentation

Index

• Constants
• type Client
  • func NewClient(url, cert, jwt string) *Client
  • func (c *Client) AddOrUpdateSet(set *set.Set) error
  • func (c *Client) GetDirs(setID string) ([]*set.Entry, error)
  • func (c *Client) GetFiles(setID string) ([]*set.Entry, error)
  • func (c *Client) GetQueueStatus() (*QStatus, error)
  • func (c *Client) GetSetByID(requester, setID string) (*set.Set, error)
  • func (c *Client) GetSetByName(requester, setName string) (*set.Set, error)
  • func (c *Client) GetSets(requester string) ([]*set.Set, error)
  • func (c *Client) GetSomeUploadRequests() ([]*put.Request, error)
  • func (c *Client) SendPutResultsToServer(results, uploadStarts chan *put.Request, minMBperSecondUploadSpeed int) error
  • func (c *Client) SetDirs(setID string, paths []string) error
  • func (c *Client) SetFiles(setID string, paths []string) error
  • func (c *Client) TriggerDiscovery(setID string) error
  • func (c *Client) UpdateFileStatus(r *put.Request) error
• type QStatus
• type Server
  • func New(logWriter io.Writer) *Server
  • func (s *Server) EnableAuth(certFile, keyFile string, acb gas.AuthCallback) error
  • func (s *Server) EnableJobSubmission(putCmd, deployment, cwd, queue string, logger log15.Logger) error
  • func (s *Server) LoadSetDB(path string) error
  • func (s *Server) QueueStatus() *QStatus

Constants

const (

	// EndPointAuthSet is the endpoint for getting and setting sets.
	EndPointAuthSet = gas.EndPointAuth + setPath

	// EndPointAuthFiles is the endpoint for setting set file paths.
	EndPointAuthFiles = gas.EndPointAuth + filePath

	// EndPointAuthDirs is the endpoint for setting set directory paths.
	EndPointAuthDirs = gas.EndPointAuth + dirPath

	// EndPointAuthEntries is the endpoint for getting set entries.
	EndPointAuthEntries = gas.EndPointAuth + entryPath

	// EndPointAuthDiscovery is the endpoint for triggering set discovery.
	EndPointAuthDiscovery = gas.EndPointAuth + discoveryPath

	// EndPointAuthRequests is the endpoint for getting file upload requests.
	EndPointAuthRequests = gas.EndPointAuth + requestsPath

	// EndPointAuthWorking is the endpoint for advising the server you're still
	// working on Requests retrieved from EndPointAuthRequests.
	EndPointAuthWorking = gas.EndPointAuth + workingPath

	// EndPointAuthQueueStatus is the endpoint for getting queue status.
	EndPointAuthQueueStatus = gas.EndPointAuth + queueStatusPath

	// EndPointAuthFileStatus is the endpoint for updating file upload status.
	EndPointAuthFileStatus = gas.EndPointAuth + fileStatusPath

	ErrNoAuth          = gas.Error("auth must be enabled")
	ErrNoSetDBDirFound = gas.Error("set database directory not found")
	ErrNoRequester     = gas.Error("requester not supplied")
	ErrBadRequester    = gas.Error("you are not the set requester")
	ErrNotAdmin        = gas.Error("you are not the server admin")
	ErrBadSet          = gas.Error("set with that id does not exist")
	ErrInvalidInput    = gas.Error("invalid input")
	ErrInteral         = gas.Error("internal server error")
)

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is used to interact with the Server over the network, with authentication.

func NewClient

func NewClient(url, cert, jwt string) *Client

NewClient returns a Client you can use to call methods on a Server listening at the given domain:port url.

Provide a non-blank path to a certificate to force us to trust that certificate, eg. if the server was started with a self-signed certificate.

You must first gas.Login() to get a JWT that you must supply here.

func (*Client) AddOrUpdateSet

func (c *Client) AddOrUpdateSet(set *set.Set) error

AddOrUpdateSet adds details about a backup set to the Server's database.

func (*Client) GetDirs

func (c *Client) GetDirs(setID string) ([]*set.Entry, error)

GetDirs gets the directories for the given set that were supplied to SetDirs().

func (*Client) GetFiles

func (c *Client) GetFiles(setID string) ([]*set.Entry, error)

GetFiles gets the defined and discovered file paths and their backup status for the given set.

func (*Client) GetQueueStatus

func (c *Client) GetQueueStatus() (*QStatus, error)

GetQueueStatus gets information about the server's queue.

func (*Client) GetSetByID

func (c *Client) GetSetByID(requester, setID string) (*set.Set, error)

GetSetByID gets details about a given requester's backup set from the Server's database. This is a convienience function that calls GetSets() and filters on the given set ID. Returns an error if the requester has no set with the given ID.

func (*Client) GetSetByName

func (c *Client) GetSetByName(requester, setName string) (*set.Set, error)

GetSetByName gets details about a given requester's backup set from the Server's database. This is a convienience function that calls GetSets() and filters on the given set name. Returns an error if the requester has no set with the given name.

func (*Client) GetSets

func (c *Client) GetSets(requester string) ([]*set.Set, error)

GetSets gets details about a given requester's backup sets from the Server's database.

func (*Client) GetSomeUploadRequests

func (c *Client) GetSomeUploadRequests() ([]*put.Request, error)

GetSomeUploadRequests gets some (approx 10GB worth of) upload Requests from the global put queue and returns them, moving from "ready" status in the queue to "running".

This automatically handles regularly telling the server knows we're still working on them, stopping when you UpdateFileStatus().

Only the user who started the server has permission to call this.

func (*Client) SendPutResultsToServer

func (c *Client) SendPutResultsToServer(results, uploadStarts chan *put.Request, minMBperSecondUploadSpeed int) error

SendPutResultsToServer reads from the given channels (as returned by put.Putter.Put()) and sends the results to the server, which will deal with any failures and update its database. Could return an error related to not being able to update the server with the results.

If an upload seems to take too long, based on the given minimum MB/s upload speed, tells the server the upload might be stuck, but continues to wait.

Do not call this concurrently!

func (*Client) SetDirs

func (c *Client) SetDirs(setID string, paths []string) error

SetDirs sets the given paths as the directory paths for the backup set with the given ID.

func (*Client) SetFiles

func (c *Client) SetFiles(setID string, paths []string) error

SetFiles sets the given paths as the file paths for the backup set with the given ID.

func (*Client) TriggerDiscovery

func (c *Client) TriggerDiscovery(setID string) error

TriggerDiscovery tells the server that you've called SetFiles() and SetDirs() for the given set, and now want it to discover the files that exist and discover the contents of the directories, and start the process of backing up the files.

func (*Client) UpdateFileStatus

func (c *Client) UpdateFileStatus(r *put.Request) error

UpdateFileStatus updates a file's status in the DB based on the given Request's status.

If the status isn't "uploading", this also tells the server we're no longer working on the request. The db update will not be carried out if we're not currently touching a corresponding request due to a prior GetSomeUploadRequests().

Only the user who started the server has permission to call this.

type QStatus

type QStatus struct {
	Total     int
	Reserved  int
	Uploading int
	Failed    int
	Stuck     []*put.Request
}

QStatus describes the status of a Server's in-memory put request queue.

type Server

type Server struct {
	gas.Server
	// contains filtered or unexported fields
}

Server is used to start a web server that provides a REST API to the setdb package's database, and a website that displays the information nicely.

func New

func New(logWriter io.Writer) *Server

New creates a Server which can serve a REST API and website.

It logs to the given io.Writer, which could for example be syslog using the log/syslog pkg with syslog.new(syslog.LOG_INFO, "tag").

func (*Server) EnableAuth

func (s *Server) EnableAuth(certFile, keyFile string, acb gas.AuthCallback) error

EnableAuth does the same as gas.EnableAuth, but also records the current username as a user with root-like permissions to work with everyone's backup sets.

func (*Server) EnableJobSubmission

func (s *Server) EnableJobSubmission(putCmd, deployment, cwd, queue string, logger log15.Logger) error

EnableJobSubmission enables submission of `ibackup put` jobs to wr in response to there being put requests from client backup sets having their discovery completed.

Supply the `ibackup put` command (ie. including absolute path to the ibackup executable and the option to get put jobs from this server).

Deployment is the wr deployment you wish to use; either 'production' or 'development'.

Added jobs will have the given cwd, which matters. If cwd is blank, the current working dir is used. If queue is not blank, that queue will be forced.

func (*Server) LoadSetDB

func (s *Server) LoadSetDB(path string) error

LoadSetDB loads the given set.db or creates it if it doesn't exist, and adds a number of endpoints to the REST API for working with the set and its entries:

GET /rest/v1/auth/status : get the global put request queue status.

PUT /rest/v1/auth/set : takes a set.Set encoded as JSON in the body to add or update details about the given set.

GET /rest/v1/auth/set/[requester] : takes "requester" URL parameter to get the sets requested by this requester.

PUT /rest/v1/auth/files/[id] : takes a []string of paths encoded as JSON in the body plus a set "id" URL parameter to store these as the files to back up for the set.

PUT /rest/v1/auth/dirs/[id] : takes a []string of paths encoded as JSON in the body plus a set "id" URL parameter to store these as the directories to recursively back up for the set.

GET /rest/v1/auth/discover/[id]: takes a set "id" URL parameter to trigger the discovery of files for the set.

GET /rest/v1/auth/entries/[id] : takes a set "id" URL parameter and returns the set.Entries with backup status about each file (both set with /rest/v1/auth/files and discovered inside /rest/v1/auth/dirs).

GET /rest/v1/auth/requests : returns about 10GB worth of upload requests from the global put queue. Only the user who started the server has permission to call this.

PUT /rest/v1/auth/working : takes a []string of Request ids encoded as JSON in the body received from the requests endpoint to advise the server you're still working on uploading those requests. Only the user who started the server has permission to call this.

PUT /rest/v1/auth/file_status : takes a put.Request encoded as JSON in the body to update the status of the corresponding set's file entry.

If the database indicates there are sets we were in the middle of working on, the upload requests will be added to our in-memory queue, just like during discovery.

You must call EnableAuth() before calling this method, and the endpoints will only let you work on sets where the Requester matches your logged-in username, or if the logged-in user is the same as the user who started the Server.

func (*Server) QueueStatus

func (s *Server) QueueStatus() *QStatus

QueueStatus returns current information about the queue's stats and any possibly stuck requests.