Documentation
¶
Index ¶
- Constants
- Variables
- func ChangeSourceString(changeSource ChangeSource) string
- func ChangeTypeString(changeType ChangeType) string
- func FindIndices(root string) ([]string, error)
- func IndexExists(baseAbs string) (bool, error)
- func ListRemote(ctx context.Context, client apigen.ClientWithResponsesInterface, loc *uri.URI, ...) error
- func WalkS3(root string, callbackFunc func(p string, info fs.FileInfo, err error) error) error
- type Change
- type ChangeSource
- type ChangeType
- type Changes
- type Index
- type MergeStrategy
- type ProgressPool
- type ProgressSpinner
- type ProgressUpdater
- type ProgressUpdaterReader
- type StringHeap
- type SyncFlags
- type SyncManager
- type Tasks
Constants ¶
const ( IndexFileName = ".lakefs_ref.yaml" IgnoreMarker = "ignored by lakectl local" IndexFileMode = 0o644 )
const ( DefaultDirectoryMask = 0o755 ClientMtimeMetadataKey = apiutil.LakeFSMetadataPrefix + "client-mtime" )
Variables ¶
Functions ¶
func ChangeSourceString ¶ added in v0.106.0
func ChangeSourceString(changeSource ChangeSource) string
func ChangeTypeString ¶ added in v0.106.0
func ChangeTypeString(changeType ChangeType) string
func FindIndices ¶
FindIndices searches the specified root directory for index files, returning their relative directory paths while skipping hidden folders.
func IndexExists ¶
func ListRemote ¶ added in v0.106.0
func ListRemote(ctx context.Context, client apigen.ClientWithResponsesInterface, loc *uri.URI, objects chan<- apigen.ObjectStats) error
ListRemote - Lists objects from a remote uri and inserts them into the objects channel
func WalkS3 ¶ added in v1.26.0
WalkS3 - walk like an Egyptian... ¯\_(ツ)¯\_ This walker function simulates the way object listing is performed by S3. Contrary to how a standard FS walk function behaves, S3 does not take into consideration the directory hierarchy. Instead, object paths include the entire path relative to the root and as a result the directory or "path separator" is also taken into account when providing the listing in a lexicographical order.
Types ¶
type Change ¶ added in v0.106.0
type Change struct { Source ChangeSource Path string Type ChangeType }
type ChangeSource ¶ added in v0.106.0
type ChangeSource int
const ( ChangeSourceRemote ChangeSource = iota ChangeSourceLocal )
type ChangeType ¶ added in v0.106.0
type ChangeType int
const ( ChangeTypeAdded ChangeType = iota ChangeTypeModified ChangeTypeRemoved ChangeTypeConflict )
func ChangeTypeFromString ¶ added in v0.106.0
func ChangeTypeFromString(changeType string) ChangeType
type Changes ¶ added in v0.106.0
type Changes []*Change
func DiffLocalWithHead ¶ added in v0.106.0
func DiffLocalWithHead(left <-chan apigen.ObjectStats, rightPath string) (Changes, error)
DiffLocalWithHead Checks changes between a local directory and the head it is pointing to. The diff check assumes the remote is an immutable set so any changes found resulted from changes in the local directory left is an object channel which contains results from a remote source. rightPath is the local directory to diff with
func Undo ¶ added in v0.106.0
Undo Creates a new list of changes that reverses the given changes list.
type Index ¶
type Index struct { PathURI string `yaml:"src"` AtHead string `yaml:"at_head"` ActiveOperation string `yaml:"active_operation"` // contains filtered or unexported fields }
Index defines the structure of the lakefs local reference file consisting of the information linking local directory with lakefs path
func WriteIndex ¶
type MergeStrategy ¶ added in v0.106.0
type MergeStrategy int
const ( MergeStrategyNone MergeStrategy = iota MergeStrategyThis MergeStrategyOther )
type ProgressPool ¶ added in v0.106.0
type ProgressPool struct {
// contains filtered or unexported fields
}
func NewProgressPool ¶ added in v0.106.0
func NewProgressPool() *ProgressPool
func (*ProgressPool) AddReader ¶ added in v0.106.0
func (p *ProgressPool) AddReader(name string, sizeBytes int64) *ProgressUpdater
func (*ProgressPool) AddSpinner ¶ added in v0.106.0
func (p *ProgressPool) AddSpinner(name string) *ProgressSpinner
func (*ProgressPool) Start ¶ added in v0.106.0
func (p *ProgressPool) Start()
func (*ProgressPool) Stop ¶ added in v0.106.0
func (p *ProgressPool) Stop()
type ProgressSpinner ¶ added in v0.106.0
type ProgressSpinner struct {
// contains filtered or unexported fields
}
func (*ProgressSpinner) Done ¶ added in v0.106.0
func (p *ProgressSpinner) Done()
func (*ProgressSpinner) Error ¶ added in v0.106.0
func (p *ProgressSpinner) Error()
type ProgressUpdater ¶ added in v0.106.0
type ProgressUpdater struct {
// contains filtered or unexported fields
}
func (*ProgressUpdater) Done ¶ added in v0.106.0
func (p *ProgressUpdater) Done()
func (*ProgressUpdater) Error ¶ added in v0.106.0
func (p *ProgressUpdater) Error()
type ProgressUpdaterReader ¶ added in v0.106.0
type ProgressUpdaterReader struct {
// contains filtered or unexported fields
}
type StringHeap ¶ added in v1.26.0
type StringHeap []string
A StringHeap is a min-heap of strings
func (*StringHeap) Len ¶ added in v1.26.0
func (pq *StringHeap) Len() int
func (*StringHeap) Less ¶ added in v1.26.0
func (pq *StringHeap) Less(i, j int) bool
func (*StringHeap) Peek ¶ added in v1.26.0
func (pq *StringHeap) Peek() any
Peek - according to heap documentation (https://pkg.go.dev/container/heap) "The minimum element in the tree is the root, at index 0."
func (*StringHeap) Pop ¶ added in v1.26.0
func (pq *StringHeap) Pop() any
func (*StringHeap) Push ¶ added in v1.26.0
func (pq *StringHeap) Push(x any)
func (*StringHeap) Swap ¶ added in v1.26.0
func (pq *StringHeap) Swap(i, j int)
type SyncManager ¶ added in v0.106.0
type SyncManager struct {
// contains filtered or unexported fields
}
func NewSyncManager ¶ added in v0.106.0
func NewSyncManager(ctx context.Context, client *apigen.ClientWithResponses, httpClient *http.Client, flags SyncFlags) *SyncManager
func (*SyncManager) Summary ¶ added in v0.106.0
func (s *SyncManager) Summary() Tasks