walk

Documentation

Overview

Package walk walks directories

Index

• Variables
• func GetAll(ctx context.Context, f fs.Fs, path string, includeAll bool, maxLevel int) (objs []fs.Object, dirs []fs.Directory, err error)
• func ListR(ctx context.Context, f fs.Fs, path string, includeAll bool, maxLevel int, ...) error
• func NewDirTree(ctx context.Context, f fs.Fs, path string, includeAll bool, maxLevel int) (dirtree.DirTree, error)
• func Walk(ctx context.Context, f fs.Fs, path string, includeAll bool, maxLevel int, ...) error
• type Func
• type ListRHelper
  • func NewListRHelper(callback fs.ListRCallback) *ListRHelper
  • func (lh *ListRHelper) Add(entry fs.DirEntry) error
  • func (lh *ListRHelper) Flush() error
• type ListType
  • func (l ListType) Dirs() bool
  • func (l ListType) Filter(in *fs.DirEntries)
  • func (l ListType) Objects() bool

Variables

var ErrorCantListR = errors.New("recursive directory listing not available")

ErrorCantListR is returned by WalkR if the underlying Fs isn't capable of doing a recursive listing.

var ErrorSkipDir = errors.New("skip this directory")

ErrorSkipDir is used as a return value from Walk to indicate that the directory named in the call is to be skipped. It is not returned as an error by any function.

Functions

func GetAll

func GetAll(ctx context.Context, f fs.Fs, path string, includeAll bool, maxLevel int) (objs []fs.Object, dirs []fs.Directory, err error)

GetAll runs ListR getting all the results

func ListR

func ListR(ctx context.Context, f fs.Fs, path string, includeAll bool, maxLevel int, listType ListType, fn fs.ListRCallback) error

ListR lists the directory recursively.

If includeAll is not set it will use the filters defined.

If maxLevel is < 0 then it will recurse indefinitely, else it will only do maxLevel levels.

If synthesizeDirs is set then for bucket based remotes it will synthesize directories from the file structure. This uses extra memory so don't set this if you don't need directories, likewise do set this if you are interested in directories.

It calls fn for each tranche of DirEntries read. Note that these don't necessarily represent a directory

Note that fn will not be called concurrently whereas the directory listing will proceed concurrently.

Directories are not listed in any particular order so you can't rely on parents coming before children or alphabetical ordering

This is implemented by using ListR on the backend if possible and efficient, otherwise by Walk.

NB (f, path) to be replaced by fs.Dir at some point

func NewDirTree

func NewDirTree(ctx context.Context, f fs.Fs, path string, includeAll bool, maxLevel int) (dirtree.DirTree, error)

NewDirTree returns a DirTree filled with the directory listing using the parameters supplied.

If includeAll is not set it will use the filters defined.

If maxLevel is < 0 then it will recurse indefinitely, else it will only do maxLevel levels.

This is implemented by WalkR if f supports ListR and level > 1, or WalkN otherwise.

If --files-from and --no-traverse is set then a DirTree will be constructed with just those files in.

NB (f, path) to be replaced by fs.Dir at some point

func Walk

func Walk(ctx context.Context, f fs.Fs, path string, includeAll bool, maxLevel int, fn Func) error

Walk lists the directory.

If includeAll is not set it will use the filters defined.

If maxLevel is < 0 then it will recurse indefinitely, else it will only do maxLevel levels.

It calls fn for each tranche of DirEntries read.

Note that fn will not be called concurrently whereas the directory listing will proceed concurrently.

Parent directories are always listed before their children

This is implemented by WalkR if Config.UseListR is true and f supports it and level > 1, or WalkN otherwise.

If --files-from and --no-traverse is set then a DirTree will be constructed with just those files in and then walked with WalkR

NB (f, path) to be replaced by fs.Dir at some point

Types

type Func

type Func func(path string, entries fs.DirEntries, err error) error

Func is the type of the function called for directory visited by Walk. The path argument contains remote path to the directory.

If there was a problem walking to directory named by path, the incoming error will describe the problem and the function can decide how to handle that error (and Walk will not descend into that directory). If an error is returned, processing stops. The sole exception is when the function returns the special value ErrorSkipDir. If the function returns ErrorSkipDir, Walk skips the directory's contents entirely.

type ListRHelper

type ListRHelper struct {
	// contains filtered or unexported fields
}

ListRHelper is used in the implementation of ListR to accumulate DirEntries

func NewListRHelper

func NewListRHelper(callback fs.ListRCallback) *ListRHelper

NewListRHelper should be called from ListR with the callback passed in

func (*ListRHelper) Add

func (lh *ListRHelper) Add(entry fs.DirEntry) error

Add an entry to the stored entries and send them if there are more than a certain amount

func (*ListRHelper) Flush

func (lh *ListRHelper) Flush() error

Flush the stored entries (if any) sending them to the callback

type ListType

type ListType byte

ListType is uses to choose which combination of files or directories is requires

const (
	ListObjects ListType                 = 1 << iota // list objects only
	ListDirs                                         // list dirs only
	ListAll     = ListObjects | ListDirs             // list files and dirs
)

Types of listing for ListR

func (ListType) Dirs

func (l ListType) Dirs() bool

Dirs returns true if the list type specifies dirs

func (ListType) Filter

func (l ListType) Filter(in *fs.DirEntries)

Filter in (inplace) to only contain the type of list entry required

func (ListType) Objects

func (l ListType) Objects() bool

Objects returns true if the list type specifies objects