reposnapshot

Documentation

Index

• type ActionOnNodeFunc
• type Node
  • func CreateNewNode(dirName string, parent *Node) *Node
  • func (node *Node) AddChildNode(dirName string, childrenPool []*Node) error
  • func (node *Node) CalculateTransferredFilesAndSize() (totalFilesCount uint32, totalFilesSize uint64, err error)
  • func (node *Node) CheckCompleted() error
  • func (node *Node) DecrementFilesCount() error
  • func (node *Node) GetChildren() (children []*Node, err error)
  • func (node *Node) IncrementFilesCount(fileSize uint64) error
  • func (node *Node) IsCompleted() (completed bool, err error)
  • func (node *Node) IsDoneExploring() (doneExploring bool, err error)
  • func (node *Node) MarkDoneExploring() error
  • func (node *Node) RestartExploring() error
• type NodeExportWrapper
• type NodeStatus
• type RepoSnapshotManager
  • func CreateRepoSnapshotManager(repoKey, snapshotFilePath string) RepoSnapshotManager
  • func LoadRepoSnapshotManager(repoKey, snapshotFilePath string) (RepoSnapshotManager, bool, error)
  • func (sm *RepoSnapshotManager) CalculateTransferredFilesAndSize() (totalFilesCount uint32, totalFilesSize uint64, err error)
  • func (sm *RepoSnapshotManager) GetDirectorySnapshotNodeWithLru(relativePath string) (node *Node, err error)
  • func (sm *RepoSnapshotManager) LookUpNode(relativePath string) (requestedNode *Node, err error)
  • func (sm *RepoSnapshotManager) PersistRepoSnapshot() error

Types

type ActionOnNodeFunc

type ActionOnNodeFunc func(node *Node) error

type Node

type Node struct {
	NodeStatus
	// contains filtered or unexported fields
}

Represents a directory in the repo state snapshot.

func CreateNewNode

func CreateNewNode(dirName string, parent *Node) *Node

func (*Node) AddChildNode

func (node *Node) AddChildNode(dirName string, childrenPool []*Node) error

Adds a new child node to children map. childrenPool - [Optional] Children array to check existence of a dirName in before creating a new node.

func (*Node) CalculateTransferredFilesAndSize

func (node *Node) CalculateTransferredFilesAndSize() (totalFilesCount uint32, totalFilesSize uint64, err error)

Sum up all subtree directories with status "completed"

func (*Node) CheckCompleted

func (node *Node) CheckCompleted() error

Check if node completed - if done exploring, done handling files, children are completed.

func (*Node) DecrementFilesCount

func (node *Node) DecrementFilesCount() error

func (*Node) GetChildren

func (node *Node) GetChildren() (children []*Node, err error)

func (*Node) IncrementFilesCount

func (node *Node) IncrementFilesCount(fileSize uint64) error

func (*Node) IsCompleted

func (node *Node) IsCompleted() (completed bool, err error)

func (*Node) IsDoneExploring

func (node *Node) IsDoneExploring() (doneExploring bool, err error)

func (*Node) MarkDoneExploring

func (node *Node) MarkDoneExploring() error

Marks that all contents of the node have been found and added.

func (*Node) RestartExploring

func (node *Node) RestartExploring() error

type NodeExportWrapper

type NodeExportWrapper struct {
	Name            string               `json:"name,omitempty"`
	Children        []*NodeExportWrapper `json:"children,omitempty"`
	Completed       bool                 `json:"completed,omitempty"`
	TotalFilesCount uint32               `json:"total_files_count,omitempty"`
	TotalFilesSize  uint64               `json:"total_files_size,omitempty"`
}

Used to export/load the node tree to/from a file. Wrapper is needed since fields on the original node are unexported (to avoid operations that aren't thread safe). The wrapper only contains fields that are used in future runs, hence not all fields from Node are persisted. In addition, it does not hold the parent pointer to avoid cyclic reference on export.

type NodeStatus

type NodeStatus uint8

const (
	Exploring NodeStatus = iota
	DoneExploring
	Completed
)

type RepoSnapshotManager

type RepoSnapshotManager struct {
	// contains filtered or unexported fields
}

Represents a snapshot of a repository being traversed to do a certain action. Each directory in the repository is represented by a node. The snapshot is constructed as a linked prefix tree, where every node has pointers to its children and parent. While traversing over the repository, contents found are added to their respecting node. Later on, files handled are removed from their node. Each node has one of three states:

1. Unexplored / Partially explored - NOT all contents of the directory were found and added to its node (Marked by NodeStatus - Exploring).
2. Fully explored - All contents of the directory were found, but NOT all of them handled (Marked by NodeStatus - DoneExploring).
3. Completed - All contents found and handled (Marked by NodeStatus - Completed).

In the event of a node reaching completion, a tree collapsing may occur in order to save space: The node will mark itself completed, and will then notify the parent to check completion as well.

When resuming from a snapshot, all nodes that weren't completed are re-explored even if they were previously fully explored. Therefore, when persisting the snapshot to disk these fields are removed.

func CreateRepoSnapshotManager

func CreateRepoSnapshotManager(repoKey, snapshotFilePath string) RepoSnapshotManager

func LoadRepoSnapshotManager

func LoadRepoSnapshotManager(repoKey, snapshotFilePath string) (RepoSnapshotManager, bool, error)

Loads a repo snapshot from the provided snapshotFilePath if such file exists. If successful, returns the snapshot and exists=true.

func (*RepoSnapshotManager) CalculateTransferredFilesAndSize

func (sm *RepoSnapshotManager) CalculateTransferredFilesAndSize() (totalFilesCount uint32, totalFilesSize uint64, err error)

Return the count and size of files that have been successfully transferred and their respective directories are marked as complete, ensuring they won't be transferred again. This data helps in estimating the remaining files for transfer after stopping.

func (*RepoSnapshotManager) GetDirectorySnapshotNodeWithLru

func (sm *RepoSnapshotManager) GetDirectorySnapshotNodeWithLru(relativePath string) (node *Node, err error)

Returns the node that represents the directory from the repo state. Updates the lru cache. relativePath - relative path of the directory.

func (*RepoSnapshotManager) LookUpNode

func (sm *RepoSnapshotManager) LookUpNode(relativePath string) (requestedNode *Node, err error)

Returns the node corresponding to the directory in the provided relative path. Path should be provided without the repository name.

func (*RepoSnapshotManager) PersistRepoSnapshot

func (sm *RepoSnapshotManager) PersistRepoSnapshot() error