proc

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 15, 2019 License: AGPL-3.0 Imports: 18 Imported by: 0

Documentation

Overview

Package proc provides actual actions to be performed once the sync has filtered all events

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MergeNodes added in v1.5.0

func MergeNodes(left *TreeNode, right *TreeNode, diff *SourceDiff)

MergeNodes will recursively detect differences between two hash trees.

Types

type ChildrenCursor added in v1.5.0

type ChildrenCursor struct {
	// contains filtered or unexported fields
}

ChildrenCursor provides a Nexter for browsing a node children

func (*ChildrenCursor) Next added in v1.5.0

func (c *ChildrenCursor) Next() *TreeNode

Next sends the next child or nil

type Merger

type Merger struct {
	BatchesChannel  chan *filters.Batch
	RequeueChannels map[PathSyncSource]chan EventInfo
	LocksChannel    chan filters.LockEvent
	UnlocksChannel  chan filters.UnlockEvent
	JobsInterrupt   chan bool
	GlobalContext   context.Context
	// contains filtered or unexported fields
}

func NewMerger

func NewMerger(ctx context.Context) *Merger

func (*Merger) AddRequeueChannel

func (b *Merger) AddRequeueChannel(source PathSyncSource, channel chan EventInfo)

func (*Merger) Logger

func (b *Merger) Logger() *zap.Logger

func (*Merger) ProcessBatches

func (b *Merger) ProcessBatches()

func (*Merger) RegisterEventChannel

func (b *Merger) RegisterEventChannel(out chan ProcessorEvent)

func (*Merger) Shutdown

func (b *Merger) Shutdown()

type ProcessFunc

type ProcessFunc func(event *filters.BatchedEvent, operationId string) error

type SourceDiff

type SourceDiff struct {
	Left         common.PathSyncSource
	Right        common.PathSyncSource
	MissingLeft  []*tree.Node
	MissingRight []*tree.Node
	Context      context.Context
}

SourceDiff represent basic differences between two sources It can be then transformed to Batch, depending on the sync being unidirectional (transform to Creates and Deletes) or bidirectional (transform only to Creates)

func ComputeSourcesDiff

func ComputeSourcesDiff(ctx context.Context, left common.PathSyncSource, right common.PathSyncSource, strong bool, statusChan chan filters.BatchProcessStatus) (diff *SourceDiff, err error)

ComputeSourceDiff loads the diff by crawling the sources in parallel, filling up a Hash Tree and performing the merge

func (*SourceDiff) FilterMissing

func (diff *SourceDiff) FilterMissing(source common.PathSyncSource, target common.PathSyncTarget, in []*tree.Node, folders bool, removes bool) (out map[string]*filters.BatchedEvent)

FilterMissing transforms Missing slices to BatchEvents

func (*SourceDiff) Stats added in v1.5.0

func (diff *SourceDiff) Stats() map[string]int

func (*SourceDiff) String

func (diff *SourceDiff) String() string

String provides a string representation of this diff

func (*SourceDiff) ToBidirectionalBatch

func (diff *SourceDiff) ToBidirectionalBatch(leftTarget common.PathSyncTarget, rightTarget common.PathSyncTarget) (batch *filters.BidirectionalBatch, err error)

ToBidirectionalBatch transforms this diff to a batch

func (*SourceDiff) ToUnidirectionalBatch

func (diff *SourceDiff) ToUnidirectionalBatch(direction string) (batch *filters.Batch, err error)

ToUnidirectionalBatch transforms this diff to a batch

type TreeNode added in v1.5.0

type TreeNode struct {
	tree.Node
	sync.Mutex
	// contains filtered or unexported fields
}

TreeNode builds a Merkle Tree but with N children and the ability to compute the COLLECTION Nodes hashes to detect changes in branches more rapidly

func NewTreeNode added in v1.5.0

func NewTreeNode(n *tree.Node) *TreeNode

NewTreeNode creates a new node from a tree.Node. Can be a root, a COLL or a LEAF.

func TreeNodeFromSource added in v1.5.0

func TreeNodeFromSource(source common.PathSyncSource) (*TreeNode, error)

TreeNodeFromSource populates a hash tree with leafs and folders by walking a source. When it comes accross a LEAF without Etag value, it asks the source to recompute it in a parallel fashion with throttling (max 15 at the same time). At the end of the operation, the tree should be fully loaded with all LEAF etags (but not COLL etags).

func (*TreeNode) AddChild added in v1.5.0

func (t *TreeNode) AddChild(n *TreeNode)

AddChild appends a child to the children map (with lock)

func (*TreeNode) Enqueue added in v1.5.0

func (t *TreeNode) Enqueue(missings []*tree.Node) []*tree.Node

Enqueue recursively appends al tree.Node and the children's tree.Node to a slice

func (*TreeNode) GetCursor added in v1.5.0

func (t *TreeNode) GetCursor() *ChildrenCursor

GetCursor gives a cursor to crawl the current node children

func (*TreeNode) GetHash added in v1.5.0

func (t *TreeNode) GetHash() string

GetHash returns the Etag of the node. For leaf it should be available, for Folders if it is not already computed, it will compute an etag from the children recursively, using their name and Etag.

func (*TreeNode) GetLevel added in v1.5.0

func (t *TreeNode) GetLevel() int

GetLevel computes the current level of this node (depth)

func (*TreeNode) Label added in v1.5.0

func (t *TreeNode) Label() string

Label returns the basename of the path

func (*TreeNode) ParentPath added in v1.5.0

func (t *TreeNode) ParentPath() string

ParentPath returns the parent Dir path

func (*TreeNode) PrintOut added in v1.5.0

func (t *TreeNode) PrintOut()

PrintOut sends to fmt.Println a tree version of this node

func (*TreeNode) SortedChildren added in v1.5.0

func (t *TreeNode) SortedChildren() []*TreeNode

SortedChildren sorts children by their labels. An internal flag avoids resorting if it was already sorted once.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL