fileutil

package
v1.10.11 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 4, 2022 License: MIT Imports: 38 Imported by: 28

Documentation

Overview

Helpers for working with files and filesystems

Index

Constants

View Source 
const DefaultOpenTimeout = time.Duration(10 * time.Second)

Variables

View Source 
var CompareHasher hash.Hash = sha256.New()
View Source 
var DirExists = pathutil.DirExists

Alias functions from pathutil as a convenience

View Source 
var Exists = pathutil.Exists
View Source 
var ExpandUser = pathutil.ExpandUser
View Source 
var FileExists = pathutil.FileExists
View Source 
var GenericDefaultTimeout = 5 * time.Second
View Source 
var HttpDefaultTimeout = 10 * time.Second
View Source 
var IsAppend = pathutil.IsAppend
View Source 
var IsAppendable = pathutil.IsAppendable
View Source 
var IsCharDevice = pathutil.IsCharDevice
View Source 
var IsDevice = pathutil.IsDevice
View Source 
var IsExclusive = pathutil.IsExclusive
View Source 
var IsNamedPipe = pathutil.IsNamedPipe
View Source 
var IsNonemptyDir = pathutil.IsNonemptyDir
View Source 
var IsNonemptyExecutableFile = pathutil.IsNonemptyExecutableFile
View Source 
var IsNonemptyFile = pathutil.IsNonemptyFile
View Source 
var IsReadable = pathutil.IsReadable
View Source 
var IsSetgid = pathutil.IsSetgid
View Source 
var IsSetuid = pathutil.IsSetuid
View Source 
var IsSocket = pathutil.IsSocket
View Source 
var IsSticky = pathutil.IsSticky
View Source 
var IsSymlink = pathutil.IsSymlink
View Source 
var IsTemporary = pathutil.IsTemporary
View Source 
var IsWritable = pathutil.IsWritable
View Source 
var LinkExists = pathutil.LinkExists
View Source 
var NullReadCloser = ioutil.NopCloser(bytes.NewBuffer(nil))
View Source 
var SkipToken = errors.New(`skip token`)

By default, the underlying bufio.Scanner that tokenizes the input data will discard the tokens that it's splitting on. however, in most cases, we don't actually want this. however, if the thing we're removing IS this token, then we will have removed the token, then immediately put it back in the stream.

Returning the SkipToken error will tell the ReadManipulator to not put this token back into the stream, but otherwise not produce an actual error during read.

View Source 
var SshDefaultTimeout = 10 * time.Second
View Source 
var SshPrivateKey = MustExpandUser(`~/.ssh/id_rsa`)
View Source 
var SshVerifyHostFunc ssh.HostKeyCallback

Functions

func AddMimeTypeFile added in v1.6.14 

func AddMimeTypeFile(filename string)

func Cat added in v1.8.83 

func Cat(file interface{}) string

Attempt to call ReadAllString, but will return an empty string if there is an error. Does not panic.

func ChecksumFile added in v1.7.7 

func ChecksumFile(filename string, fn interface{}) ([]byte, error)

func CompareReaders added in v1.8.20 

func CompareReaders(a io.Reader, b io.Reader, hasher ...hash.Hash) int

Compare the binary contents of two io.Reader instances. The result will be 0 if a==b, -1 if a < b, and +1 if a > b. An optional hash.Hash instance may be given, otherwise the default crypto/sha256 will be used.

func CopyDir added in v1.6.33 

func CopyDir(root string, fn CopyEntryFunc) error

Recursively walk the entries of a given directory, calling CopyEntryFunc for each entry. The io.Writer returned from the function will have that file's contents written to it. If the io.Writer is nil, the file will not be written anywhere but no error will be returned. If CopyEntryFunc returns an error, the behavior will be consistent with filepath.WalkFunc

func CopyFile added in v1.7.17 

func CopyFile(source interface{}, destination interface{}) error

Copy a file from one place to another. Source can be an io.Reader or string. If source is a string, the string will be passed to the Open() function as a URL. Destination can be an io.Writer or string. If destination is a string, it will be treated as a local filesystem path to write the data read from source to.

If either source or destination implements io.Closer, thee files will be closed before this function returns.

func GetMimeType added in v1.6.14 

func GetMimeType(filenameOrReader interface{}, fallback ...string) string

GetMimeType goes out of its way to really, really try to figure out the MIME type of a given filename or io.Reader. If the first argument satisfies the io.Seeker interface, the seeker will Seek() back to the beginning. If it satisfies the io.Closer interface, it will be closed.

func GetNthLine added in v1.8.35 

func GetNthLine(file interface{}, number int) (string, error)

Attempt to return the nth line (starting from 1) in the given file or reader.

func InitMime added in v1.6.14 

func InitMime()

func IsModifiedAfter added in v1.6.27 

func IsModifiedAfter(stat os.FileInfo, current string) bool

Return true if the given FileInfo sports a ModTime later than the current file.

func IsTerminal 

func IsTerminal() bool

func MustExpandUser added in v1.6.27 

func MustExpandUser(path string) string

func MustGetNthLine added in v1.8.35 

func MustGetNthLine(file interface{}, number int) string

A panicky version of GetNthLine.

func MustReadAll 

func MustReadAll(file interface{}) []byte

func MustReadAllLines added in v1.8.35 

func MustReadAllLines(file interface{}) []string

A panicky version of ReadAllLines.

func MustReadAllString 

func MustReadAllString(file interface{}) string

func MustWriteFile added in v1.6.26 

func MustWriteFile(input interface{}, filename string) int64

Same as WriteFile, but will panic if the file cannot be written.

func MustWriteTempFile added in v1.7.30 

func MustWriteTempFile(input interface{}, prefix string) string

Same as MustWriteFile, but writes the given input to a temporary file, returning the filename. The function will panic if the file cannot be written.

func Open added in v1.7.17 

func Open(uri string) (io.Reader, error)

Calls OpenWithOptions with no options set.

func OpenWithOptions added in v1.7.17 

func OpenWithOptions(uri string, options OpenOptions) (io.Reader, error)

A generic URL opener that supports various schemes and returns an io.Reader. Supported URL schemes include: file://, http://, https://, ftp://, sftp://. If no scheme is provided, the URL is interpreted as a local filesystem path.

func ReadAll 

func ReadAll(file interface{}) ([]byte, error)

Takes the given string filename, []byte, io.Reader, or io.ReadCloser and returns the bytes therein.

func ReadAllLines added in v1.6.13 

func ReadAllLines(file interface{}) ([]string, error)

Read all lines of text from the given file and return them as a slice.

func ReadAllString 

func ReadAllString(file interface{}) (string, error)

A string version of ReadAll.

func ReadFirstLine added in v1.6.13 

func ReadFirstLine(file interface{}) (string, error)

func RegisterOpenHandler added in v1.7.17 

func RegisterOpenHandler(scheme string, handler OpenHandler)

Register a handler for a new or existing URL scheme, for use with Open() and OpenWithOptions()

func RegisterRetrieveScheme added in v1.9.0 

func RegisterRetrieveScheme(scheme string, fn RetrieveHandlerFunc)

Register a new Retrieve() function for the given URL scheme.

func RemoveBlankLines added in v1.7.54 

func RemoveBlankLines(data []byte) ([]byte, error)

A ReadManipulatorFunc for removing lines that only contain whitespace.

func RemoveOpenHandler added in v1.7.17 

func RemoveOpenHandler(scheme string)

Removes a registered URL scheme handler.

func Retrieve added in v1.9.0 

func Retrieve(ctx context.Context, resourceUri interface{}) (io.ReadCloser, error)

Perform a generic retrieval of data located at a specified resource given as a URL. This function supports file://, http://, https://, ssh://, and sftp:// schemes, and can be extended to support additional schemes using the RegisterRetrieveScheme package function.

If resourceUri is given as a *url.URL, the value of that URL will be copied. Any other type will be converted to a string (honoring types that implement fmt.Stringer), and the resulting URL will be used.

func RetrieveViaFilesystem added in v1.9.0 

func RetrieveViaFilesystem(ctx context.Context, u *url.URL) (io.ReadCloser, error)

Retrieve a file via a filesystem. If the context value `filesystem` implements the http.FileSystem interface, it will be used to perform the retrieval in lieu of the local filesystem.

func RetrieveViaHTTP added in v1.9.0 

func RetrieveViaHTTP(ctx context.Context, u *url.URL) (io.ReadCloser, error)

Retrieve a file via HTTP or HTTPS.

Supported Context Values: 

insecure:
  (bool) specify that strict TLS validation should be optional.

method:
  (string) the HTTP method to use, defaults to GET.

metadata:
  (map[string]interface{}) a key-value set of HTTP request headers to include.

safeResponseCodes:
  ([]int) a list of one or more HTTP status codes that are considered successful for this request.

func RetrieveViaSSH added in v1.9.0 

func RetrieveViaSSH(ctx context.Context, u *url.URL) (io.ReadCloser, error)

Retrieve a file via SFTP (SSH file transfer). The given URL should resemble the prototype: ssh://[user:password@]hostname[:22]/path/relative/to/homedir

This function will honor any authentication details from a running SSH agent, as well as utilize the private key located in the path indicated by SshPrivateKey or via the `privateKey` context value.

Supported Context Values:

username: 

(string) the username to login with. can be overriden by a username specified in the URL.

password: 

(string) the password to login with. can be overriden by a password specified in the URL.

passphrase: 

(string) context value specifies a plaintext passphrase used to unlock the local private keyfile.

insecure: 

(bool) whether to ignore remote hostkey checks.  Does not work if verifyHostFunc is set.

verifyHostFunc: 

(SshHostKeyCallbackFunc) context value, if it is convertible to the ssh.HostKeyCallback type, will
be called to verify the remote SSH host key in a manner of the function's choosing.  The default
behavior is to accept all remote hostkeys as valid.

func SameFile added in v1.8.20 

func SameFile(first interface{}, second interface{}) bool

Returns whether two files represent the same file. If a string is given for either file, os.Stat will be run on that path. If os.FileInfo is given for either file, it will be passed to os.SameFile directly. If either file is an io.Reader, the contents of both files will be read and hashed using CompareHasher. If the hashes are identical, the files are considered the same. Any error encountered and this function will return false.

func SetExt added in v1.8.16 

func SetExt(path string, ext string, oldexts ...string) string

Detects the file extension of the given path and replaces it with the given extension. The optional second argument allows you to explictly specify the extension (if known).

func ShouldGetNthLine added in v1.8.35 

func ShouldGetNthLine(file interface{}, number int) string

Attempts to call GetNthLine, but will return an empty string if there is an error. Does not panic.

func ShouldReadAllLines added in v1.8.35 

func ShouldReadAllLines(file interface{}) []string

Attempts to call ReadAllLines, but will return an empty slice if there is an error. Does not panic.

func SizeOf added in v1.8.19 

func SizeOf(path string) convutil.Bytes

Returns the size of the given filename, or zero if any error is encountered (or if the file is zero length).

func UnregisterRetrieveScheme added in v1.9.2 

func UnregisterRetrieveScheme(scheme string)

Unregister the given URL scheme from being handled by the Retrieve() function.

func WriteFile added in v1.6.26 

func WriteFile(input interface{}, filename string) (int64, error)

Write the contents of the given io.Reader, []byte, or string to the specified filename. Filename paths containing tilde (~) will automatically expand to the current user's home directory, and all intermediate parent directories will be automatically created. Will return the number of bytes written, or an error.

func WriteTempFile added in v1.7.30 

func WriteTempFile(input interface{}, pattern string) (string, error)

Same as WriteFile, but writes the given input to a temporary file, returning the filename.

Types

type CloserFunc added in v1.9.0 

type CloserFunc = func(io.ReadCloser) error

type CopyEntryFunc added in v1.6.33 

type CopyEntryFunc func(path string, info os.FileInfo, err error) (io.Writer, error)

type DirReader added in v1.6.32 

type DirReader struct {
	// contains filtered or unexported fields
}

A DirReader provides a streaming io.Reader interface to all files in a given directory, with options for handling unreadable entries and recursion.

func NewDirReader added in v1.6.32 

func NewDirReader(path string, options ...DirReaderOption) *DirReader

func (*DirReader) Close added in v1.6.32 

func (self *DirReader) Close() error

close open files and reset the internal reader

func (*DirReader) Read added in v1.6.32 

func (self *DirReader) Read(b []byte) (int, error)

func (*DirReader) SetSkipFunc added in v1.6.32 

func (self *DirReader) SetSkipFunc(fn SkipFunc)

Set a function that will be called for each path encountered while reading. If this function returns true, that path (and its descedants) will not be read.

type DirReaderOption added in v1.6.32 

type DirReaderOption int
const (
	NoRecursive DirReaderOption = iota
	FailOnError
)

type DirReaderOptions added in v1.6.32 

type DirReaderOptions []DirReaderOption

func (DirReaderOptions) Has added in v1.6.32 

func (self DirReaderOptions) Has(option DirReaderOption) bool

type ExtendableReader added in v1.9.11 

type ExtendableReader struct {
	// contains filtered or unexported fields
}

func (*ExtendableReader) AppendSource added in v1.9.11 

func (self *ExtendableReader) AppendSource(rc io.ReadCloser)

func (*ExtendableReader) Close added in v1.9.11 

func (self *ExtendableReader) Close() error

func (*ExtendableReader) Read added in v1.9.11 

func (self *ExtendableReader) Read(b []byte) (int, error)

type FileInfo added in v1.7.44 

type FileInfo struct {
	os.FileInfo
	// contains filtered or unexported fields
}

An os.FileInfo-compatible wrapper that allows for individual values to be overridden.

func NewFileInfo added in v1.7.44 

func NewFileInfo(wrap ...os.FileInfo) *FileInfo

func (*FileInfo) IsDir added in v1.7.44 

func (self *FileInfo) IsDir() bool

func (*FileInfo) ModTime added in v1.7.44 

func (self *FileInfo) ModTime() time.Time

func (*FileInfo) Mode added in v1.7.44 

func (self *FileInfo) Mode() os.FileMode

func (*FileInfo) Name added in v1.7.44 

func (self *FileInfo) Name() string

func (*FileInfo) SetIsDir added in v1.7.44 

func (self *FileInfo) SetIsDir(isDir bool)

func (*FileInfo) SetModTime added in v1.7.44 

func (self *FileInfo) SetModTime(mtime time.Time)

func (*FileInfo) SetMode added in v1.7.44 

func (self *FileInfo) SetMode(mode os.FileMode)

func (*FileInfo) SetName added in v1.7.44 

func (self *FileInfo) SetName(name string)

func (*FileInfo) SetSize added in v1.7.44 

func (self *FileInfo) SetSize(sz int64)

func (*FileInfo) SetSys added in v1.7.44 

func (self *FileInfo) SetSys(iface interface{})

func (*FileInfo) Size added in v1.7.44 

func (self *FileInfo) Size() int64

func (*FileInfo) Sys added in v1.7.44 

func (self *FileInfo) Sys() interface{}

type MultiCloser added in v1.10.8 

type MultiCloser struct {
	// contains filtered or unexported fields
}

func NewMultiCloser added in v1.10.8 

func NewMultiCloser(closers ...io.Closer) *MultiCloser

func (*MultiCloser) Close added in v1.10.8 

func (self *MultiCloser) Close() (merr error)

type OpenHandler added in v1.7.17 

type OpenHandler func(*url.URL, OpenOptions) (io.ReadCloser, error)

type OpenOptions added in v1.7.17 

type OpenOptions struct {
	Timeout  time.Duration
	Insecure bool
}

func (OpenOptions) GetTimeout added in v1.7.17 

func (self OpenOptions) GetTimeout() time.Duration

type PostReadCloser added in v1.7.17 

type PostReadCloser struct {
	// contains filtered or unexported fields
}

func NewPostReadCloser added in v1.7.17 

func NewPostReadCloser(upstream io.ReadCloser, closer CloserFunc) *PostReadCloser

Implements an io.ReadCloser that can be configured to perform cleanup options whenever the Close() function is called. If CloserFunc is non-nil, it will be given the upstream ReadCloser as an argument and will be responsible for calling Close() on it. If nil, upstream's Close() function will be called directly on Close.

func (*PostReadCloser) Close added in v1.7.17 

func (self *PostReadCloser) Close() error

func (*PostReadCloser) Read added in v1.7.17 

func (self *PostReadCloser) Read(b []byte) (int, error)

type ReadManipulator added in v1.7.54 

type ReadManipulator struct {
	// contains filtered or unexported fields
}

func NewReadManipulator added in v1.7.54 

func NewReadManipulator(reader io.Reader, fns ...ReadManipulatorFunc) *ReadManipulator

func (*ReadManipulator) Close added in v1.7.54 

func (self *ReadManipulator) Close() error

func (*ReadManipulator) Read added in v1.7.54 

func (self *ReadManipulator) Read(b []byte) (int, error)

func (*ReadManipulator) Split added in v1.7.54 

func (self *ReadManipulator) Split(split bufio.SplitFunc)

type ReadManipulatorFunc added in v1.7.54 

type ReadManipulatorFunc func(data []byte) ([]byte, error)

func ManipulateAll added in v1.7.54 

func ManipulateAll(fns ...ReadManipulatorFunc) ReadManipulatorFunc

Performs multiple sequential manipulations on an intercepted line of text from an io.Reader as its being read.

func RemoveLinesContaining added in v1.7.54 

func RemoveLinesContaining(needle string) ReadManipulatorFunc

A ReadManipulatorFunc for removing lines that contain the given string.

func RemoveLinesSurroundedBy added in v1.7.54 

func RemoveLinesSurroundedBy(prefix string, suffix string, trimSpace bool) ReadManipulatorFunc

A ReadManipulatorFunc for removing lines surrounded by a given prefix and suffix.

func RemoveLinesWithPrefix added in v1.7.54 

func RemoveLinesWithPrefix(prefix string, trimSpace bool) ReadManipulatorFunc

A ReadManipulatorFunc for removing lines that have a given prefix.

func RemoveLinesWithSuffix added in v1.7.54 

func RemoveLinesWithSuffix(suffix string, trimSpace bool) ReadManipulatorFunc

A ReadManipulatorFunc for removing lines that have a given suffix.

func ReplaceWith added in v1.7.54 

func ReplaceWith(find string, replace string, occurrences int) ReadManipulatorFunc

A ReadManipulatorFunc for replacing text in an io.Reader as its being read.

type RetrieveHandlerFunc added in v1.9.0 

type RetrieveHandlerFunc = func(context.Context, *url.URL) (io.ReadCloser, error)

type RewriteFileSystem 

type RewriteFileSystem struct {
	FileSystem http.FileSystem
	Find       *regexp.Regexp
	Replace    string
	MustMatch  bool
}

func (RewriteFileSystem) Open 

func (self RewriteFileSystem) Open(name string) (http.File, error)

type SkipFunc added in v1.6.32 

type SkipFunc func(string) bool

type SshHostKeyCallbackFunc added in v1.9.2 

type SshHostKeyCallbackFunc = func(hostname string, remote net.Addr, key ssh.PublicKey) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.