file

package module
v0.0.0-...-deda848 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 14, 2025 License: Apache-2.0 Imports: 9 Imported by: 7

README

Package cloudeng.io/file

import cloudeng.io/file

Variables

ErrNotImplemented
ErrNotImplemented = fmt.Errorf("not implemented")

ErrNotImplemented is returned by methods that are not implemented by a particular filesystem.

Functions

Func ContextWithFS
func ContextWithFS(ctx context.Context, container ...fs.ReadFileFS) context.Context

ContextWithFS returns a new context that contains the provided instances of fs.ReadFileFS stored with as a value within it.

Func FSFromContext
func FSFromContext(ctx context.Context) ([]fs.ReadFileFS, bool)

FSFromContext returns the list of fs.ReadFileFS instancees, if any, stored within the context.

Func FSOpen
func FSOpen(ctx context.Context, filename string) (fs.File, error)

FSOpen will attempt to open filename using the context's set of fs.ReadFileFS instances (if any), in the order in which they were provided to ContextWithFS, returning the first successful result. If no fs.ReadFileFS instances are present in the context or none successfully open the file, then os.Open is used.

Func FSReadFile
func FSReadFile(ctx context.Context, name string) ([]byte, error)

FSreadFile is like FSOpen but calls ReadFile instead of Open.

Types

Type FS
type FS interface {
	fs.FS

	// Scheme returns the URI scheme that this FS supports. Scheme should
	// be "file" for local file system access.
	Scheme() string

	// OpenCtx is like fs.Open but with a context.
	OpenCtx(ctx context.Context, name string) (fs.File, error)

	// Readlink returns the contents of a symbolic link.
	Readlink(ctx context.Context, path string) (string, error)

	// Stat will follow symlinks/redirects/aliases.
	Stat(ctx context.Context, path string) (Info, error)

	// Lstat will not follow symlinks/redirects/aliases.
	Lstat(ctx context.Context, path string) (Info, error)

	// Join is like filepath.Join for the filesystem supported by this filesystem.
	Join(components ...string) string

	// Base is like filepath.Base for the filesystem supported by this filesystem.
	Base(path string) string

	// IsPermissionError returns true if the specified error, as returned
	// by the filesystem's implementation, is a result of a permissions error.
	IsPermissionError(err error) bool

	// IsNotExist returns true if the specified error, as returned by the
	// filesystem's implementation, is a result of the object not existing.
	IsNotExist(err error) bool

	// XAttr returns extended attributes for the specified file.Info
	// and file.
	XAttr(ctx context.Context, path string, fi Info) (XAttr, error)

	// SysXAttr returns a representation of the extended attributes using the
	// native data type of the underlying file system. If existing is
	// non-nil and is of that file-system specific type the contents of
	// XAttr are merged into it.
	SysXAttr(existing any, merge XAttr) any
}

FS extends fs.FS with Scheme and OpenCtx.

Type Info
type Info struct {
	// contains filtered or unexported fields
}

Info implements fs.FileInfo to provide binary, gob and json encoding/decoding. The SysInfo field is not encoded/decoded and hence is only available for use within the process that Info was instantiated in.

Functions
func NewInfo(
	name string,
	size int64,
	mode fs.FileMode,
	modTime time.Time,
	sysInfo any) Info

NewInfo creates a new instance of Info.

func NewInfoFromFileInfo(fi fs.FileInfo) Info

NewInfoFromFileInfo creates a new instance of Info from a fs.FileInfo.

Methods
func (fi *Info) AppendBinary(buf *bytes.Buffer) error

AppendBinary appends a binary encoded instance of Info to the supplied buffer.

func (fi *Info) DecodeBinary(data []byte) ([]byte, error)

DecodeBinary decodes the supplied data into the receiver and returns the remaining data.

func (fi Info) IsDir() bool

IsDir implements fs.FileInfo.

func (fi Info) MarshalBinary() ([]byte, error)

Implements encoding.BinaryMarshaler.

func (fi Info) MarshalJSON() ([]byte, error)

func (fi Info) ModTime() time.Time

ModTime implements fs.FileInfo.

func (fi Info) Mode() fs.FileMode

Mode implements fs.FileInfo.

func (fi Info) Name() string

Name implements fs.FileInfo.

func (fi *Info) SetSys(i any)

SetSys sets the SysInfo field. Note that the Sys field is never encoded/decoded.

func (fi Info) Size() int64

Size implements fs.FileInfo.

func (fi Info) Sys() any

Sys implements fs.FileInfo. Note that the Sys field is never encoded/decoded.

func (fi Info) Type() fs.FileMode

Type implements fs.Entry

func (fi *Info) UnmarshalBinary(data []byte) error

Implements encoding.BinaryUnmarshaler.

func (fi *Info) UnmarshalJSON(data []byte) error
Type InfoList
type InfoList []Info

InfoList represents a list of Info instances. It provides efficient encoding/decoding operations.

Functions
func DecodeBinaryInfoList(data []byte) (InfoList, []byte, error)

DecodeBinaryInfoList decodes the supplied data into an InfoList and returns the remaining data.

Methods
func (il InfoList) AppendBinary(buf *bytes.Buffer) error

AppendBinary appends a binary encoded instance of Info to the supplied byte slice.

func (il InfoList) AppendInfo(info Info) InfoList

Append appends an Info instance to the list and returns the updated list.

func (il InfoList) MarshalBinary() ([]byte, error)

MarshalBinary implements encoding.BinaryMarshaler.

func (il *InfoList) UnmarshalBinary(data []byte) (err error)

UnmarshalBinary implements encoding.BinaryUnmarshaler.

Type ObjectFS
type ObjectFS interface {
	Get(ctx context.Context, path string) ([]byte, error)
	Put(ctx context.Context, path string, perm fs.FileMode, data []byte) error
	EnsurePrefix(ctx context.Context, path string, perm fs.FileMode) error
	Delete(ctx context.Context, path string) error
	// DeleteAll delets all objects with the specified prefix.
	DeleteAll(ctx context.Context, prefix string) error
}

ObjectFS represents a writeable object store. It is intended to backed by cloud or local filesystems. The permissions may be ignored by some implementations.

Type XAttr
type XAttr struct {
	UID, GID       int64  // -1 for non-posix filesystems that don't support numeric UID, GID
	User, Group    string // Used for systems that don't support numeric UID, GID
	Device, FileID uint64
	Blocks         int64
	Hardlinks      uint64
}

XAttr represents extended information about a directory or file as obtained from the filesystem.

Methods
func (x XAttr) CompareGroup(o XAttr) bool

CompareGroup compares the GID fields if >=0 and the Group fields otherwise.

func (x XAttr) CompareUser(o XAttr) bool

CompareUser compares the UID fields if >=0 and the User fields otherwise.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrNotImplemented = fmt.Errorf("not implemented")

ErrNotImplemented is returned by methods that are not implemented by a particular filesystem.

Functions

func ContextWithFS 

func ContextWithFS(ctx context.Context, container ...fs.ReadFileFS) context.Context

ContextWithFS returns a new context that contains the provided instances of fs.ReadFileFS stored with as a value within it.

func FSFromContext 

func FSFromContext(ctx context.Context) ([]fs.ReadFileFS, bool)

FSFromContext returns the list of fs.ReadFileFS instancees, if any, stored within the context.

func FSOpen 

func FSOpen(ctx context.Context, filename string) (fs.File, error)

FSOpen will attempt to open filename using the context's set of fs.ReadFileFS instances (if any), in the order in which they were provided to ContextWithFS, returning the first successful result. If no fs.ReadFileFS instances are present in the context or none successfully open the file, then os.Open is used.

func FSReadFile 

func FSReadFile(ctx context.Context, name string) ([]byte, error)

FSreadFile is like FSOpen but calls ReadFile instead of Open.

Types

type FS 

type FS interface {
	fs.FS

	// Scheme returns the URI scheme that this FS supports. Scheme should
	// be "file" for local file system access.
	Scheme() string

	// OpenCtx is like fs.Open but with a context.
	OpenCtx(ctx context.Context, name string) (fs.File, error)

	// Readlink returns the contents of a symbolic link.
	Readlink(ctx context.Context, path string) (string, error)

	// Stat will follow symlinks/redirects/aliases.
	Stat(ctx context.Context, path string) (Info, error)

	// Lstat will not follow symlinks/redirects/aliases.
	Lstat(ctx context.Context, path string) (Info, error)

	// Join is like filepath.Join for the filesystem supported by this filesystem.
	Join(components ...string) string

	// Base is like filepath.Base for the filesystem supported by this filesystem.
	Base(path string) string

	// IsPermissionError returns true if the specified error, as returned
	// by the filesystem's implementation, is a result of a permissions error.
	IsPermissionError(err error) bool

	// IsNotExist returns true if the specified error, as returned by the
	// filesystem's implementation, is a result of the object not existing.
	IsNotExist(err error) bool

	// XAttr returns extended attributes for the specified file.Info
	// and file.
	XAttr(ctx context.Context, path string, fi Info) (XAttr, error)

	// SysXAttr returns a representation of the extended attributes using the
	// native data type of the underlying file system. If existing is
	// non-nil and is of that file-system specific type the contents of
	// XAttr are merged into it.
	SysXAttr(existing any, merge XAttr) any
}

FS extends fs.FS with Scheme and OpenCtx.

type Info 

type Info struct {
	// contains filtered or unexported fields
}

Info implements fs.FileInfo to provide binary, gob and json encoding/decoding. The SysInfo field is not encoded/decoded and hence is only available for use within the process that Info was instantiated in.

func NewInfo 

func NewInfo(
	name string,
	size int64,
	mode fs.FileMode,
	modTime time.Time,
	sysInfo any) Info

NewInfo creates a new instance of Info.

func NewInfoFromFileInfo 

func NewInfoFromFileInfo(fi fs.FileInfo) Info

NewInfoFromFileInfo creates a new instance of Info from a fs.FileInfo.

func (*Info) AppendBinary 

func (fi *Info) AppendBinary(buf *bytes.Buffer) error

AppendBinary appends a binary encoded instance of Info to the supplied buffer.

func (*Info) DecodeBinary 

func (fi *Info) DecodeBinary(data []byte) ([]byte, error)

DecodeBinary decodes the supplied data into the receiver and returns the remaining data.

func (Info) IsDir 

func (fi Info) IsDir() bool

IsDir implements fs.FileInfo.

func (Info) MarshalBinary 

func (fi Info) MarshalBinary() ([]byte, error)

Implements encoding.BinaryMarshaler.

func (Info) MarshalJSON 

func (fi Info) MarshalJSON() ([]byte, error)

func (Info) ModTime 

func (fi Info) ModTime() time.Time

ModTime implements fs.FileInfo.

func (Info) Mode 

func (fi Info) Mode() fs.FileMode

Mode implements fs.FileInfo.

func (Info) Name 

func (fi Info) Name() string

Name implements fs.FileInfo.

func (*Info) SetSys 

func (fi *Info) SetSys(i any)

SetSys sets the SysInfo field. Note that the Sys field is never encoded/decoded.

func (Info) Size 

func (fi Info) Size() int64

Size implements fs.FileInfo.

func (Info) Sys 

func (fi Info) Sys() any

Sys implements fs.FileInfo. Note that the Sys field is never encoded/decoded.

func (Info) Type 

func (fi Info) Type() fs.FileMode

Type implements fs.Entry

func (*Info) UnmarshalBinary 

func (fi *Info) UnmarshalBinary(data []byte) error

Implements encoding.BinaryUnmarshaler.

func (*Info) UnmarshalJSON 

func (fi *Info) UnmarshalJSON(data []byte) error

type InfoList 

type InfoList []Info

InfoList represents a list of Info instances. It provides efficient encoding/decoding operations.

func DecodeBinaryInfoList 

func DecodeBinaryInfoList(data []byte) (InfoList, []byte, error)

DecodeBinaryInfoList decodes the supplied data into an InfoList and returns the remaining data.

func (InfoList) AppendBinary 

func (il InfoList) AppendBinary(buf *bytes.Buffer) error

AppendBinary appends a binary encoded instance of Info to the supplied byte slice.

func (InfoList) AppendInfo 

func (il InfoList) AppendInfo(info Info) InfoList

Append appends an Info instance to the list and returns the updated list.

func (InfoList) MarshalBinary 

func (il InfoList) MarshalBinary() ([]byte, error)

MarshalBinary implements encoding.BinaryMarshaler.

func (*InfoList) UnmarshalBinary 

func (il *InfoList) UnmarshalBinary(data []byte) (err error)

UnmarshalBinary implements encoding.BinaryUnmarshaler.

type ObjectFS 

type ObjectFS interface {
	Get(ctx context.Context, path string) ([]byte, error)
	Put(ctx context.Context, path string, perm fs.FileMode, data []byte) error
	EnsurePrefix(ctx context.Context, path string, perm fs.FileMode) error
	Delete(ctx context.Context, path string) error
	// DeleteAll delets all objects with the specified prefix.
	DeleteAll(ctx context.Context, prefix string) error
}

ObjectFS represents a writeable object store. It is intended to backed by cloud or local filesystems. The permissions may be ignored by some implementations.

type XAttr 

type XAttr struct {
	UID, GID       int64  // -1 for non-posix filesystems that don't support numeric UID, GID
	User, Group    string // Used for systems that don't support numeric UID, GID
	Device, FileID uint64
	Blocks         int64
	Hardlinks      uint64
}

XAttr represents extended information about a directory or file as obtained from the filesystem.

func (XAttr) CompareGroup 

func (x XAttr) CompareGroup(o XAttr) bool

CompareGroup compares the GID fields if >=0 and the Group fields otherwise.

func (XAttr) CompareUser 

func (x XAttr) CompareUser(o XAttr) bool

CompareUser compares the UID fields if >=0 and the User fields otherwise.

Source Files

View all Source files

Directories

Path Synopsis
Package checkpoint provides a mechanism for checkpointing the state of an ongoing operation.
 Package checkpoint provides a mechanism for checkpointing the state of an ongoing operation.
Package content provides support for working with different content types.
 Package content provides support for working with different content types.
internal
processors
Package processor provides support for processing different content types.
 Package processor provides support for processing different content types.
stores
Package crawl provides a framework for multilevel/recursive crawling files.
 Package crawl provides a framework for multilevel/recursive crawling files.
crawlcmd
Package crawlcmd provides support for building command line tools for crawling.
 Package crawlcmd provides support for building command line tools for crawling.
outlinks
Package download provides a simple download mechanism that uses the fs.FS container API to implement the actual download.
 Package download provides a simple download mechanism that uses the fs.FS container API to implement the actual download.
Package filewalk provides support for concurrent traversal of file system directories and files.
 Package filewalk provides support for concurrent traversal of file system directories and files.
asyncstat
filewalktestutil
Package filewalktestutil provides utilities for testing code that uses filewalk.FS.
 Package filewalktestutil provides utilities for testing code that uses filewalk.FS.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.