sync

Documentation

Overview

Package sync provides utility functions similar to `git pull/push` for PFS

Index

• func Push(client *pachclient.APIClient, root string, commit *pfs.Commit, overwrite bool) error
• func PushFile(c *pachclient.APIClient, pfc pachclient.PutFileClient, pfsFile *pfs.File, ...) error
• func PushObj(pachClient *pachclient.APIClient, commit *pfs.Commit, objClient obj.Client, ...) error
• type Puller
  • func NewPuller() *Puller
  • func (p *Puller) CleanUp() (int64, error)
  • func (p *Puller) Pull(client *pachclient.APIClient, root string, repo, commit, file string, ...) error
  • func (p *Puller) PullDiff(client *pachclient.APIClient, root string, ...) error
  • func (p *Puller) PullTree(client *pachclient.APIClient, root string, tree hashtree.HashTree, pipes bool, ...) error

Functions

func Push

func Push(client *pachclient.APIClient, root string, commit *pfs.Commit, overwrite bool) error

Push puts files under root into an open commit.

func PushFile

func PushFile(c *pachclient.APIClient, pfc pachclient.PutFileClient, pfsFile *pfs.File, osFile io.ReadSeeker) error

PushFile makes sure that pfsFile has the same content as osFile.

func PushObj

func PushObj(pachClient *pachclient.APIClient, commit *pfs.Commit, objClient obj.Client, root string) error

PushObj pushes data from commit to an object store.

Types

type Puller

type Puller struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Puller as a struct for managing a Pull operation.

func NewPuller

func NewPuller() *Puller

NewPuller creates a new Puller struct.

func (*Puller) CleanUp

func (p *Puller) CleanUp() (int64, error)

CleanUp cleans up blocked syscalls for pipes that were never opened. And returns the total number of bytes that have been pulled/pushed. It also returns any errors that might have been encountered while trying to read data for the pipes. CleanUp should be called after all code that might access pipes has completed running, it should not be called concurrently.

func (*Puller) Pull

func (p *Puller) Pull(client *pachclient.APIClient, root string, repo, commit, file string,
	pipes bool, emptyFiles bool, concurrency int, statsTree *hashtree.Ordered, statsRoot string) error

Pull clones an entire repo at a certain commit. root is the local path you want to clone to. repo, commit, file specify the file/dir we are pulling. pipes causes the function to create named pipes in place of files, thus lazily downloading the data as it's needed. emptyFiles causes the function to create empty files with no content, it's mutually exclusive with pipes. tree is a hashtree to mirror the pulled content into (it may be left nil) treeRoot is the root the data is mirrored to within tree

func (*Puller) PullDiff

func (p *Puller) PullDiff(client *pachclient.APIClient, root string, newRepo, newCommit, newPath, oldRepo, oldCommit, oldPath string,
	newOnly bool, pipes bool, emptyFiles bool, concurrency int, tree hashtree.HashTree, treeRoot string) error

PullDiff is like Pull except that it materializes a Diff of the content rather than a the actual content. If newOnly is true then only new files will be downloaded and they will be downloaded under root. Otherwise new and old files will be downloaded under root/new and root/old respectively.

func (*Puller) PullTree

func (p *Puller) PullTree(client *pachclient.APIClient, root string, tree hashtree.HashTree, pipes bool, concurrency int) error

PullTree pulls from a raw HashTree rather than a repo.