rifs

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2020 License: MIT Imports: 7 Imported by: 1

README

GoDoc Build Status Coverage Status Go Report Card

bounceback_reader, bounceback_writer

An io.ReadSeeker and io.WriteSeeker that returns to the right place before reading or writing. Useful when the same file resource is being reused for reads or writes throughout that file.

list_files

A recursive path walker that supports filters.

seekable_buffer

A memory structure that satisfies io.ReadWriteSeeker.

copy_bytes_between_positions

Given an io.ReadWriteSeeker, copy N bytes from one position to an earlier position.

read_counter, write_counter

Wrap io.Reader and io.Writer structs in order to report how many bytes were transferred.

readseekwritecloser

Provides the ReadWriteSeekCloser interface that combines a RWS and a Closer. Also provides a no-op wrapper to augment a plain RWS with a closer.

bounedreadwriteseek

Wraps a ReadWriteSeeker such that no seeks can be at an offset less than a specific-offset.

calculateseek

Provides a reusable function with which to calculate seek offsets.

progress_wrapper

Provides io.Reader and io.Writer wrappers that also trigger callbacks after each call. The reader wrapper also invokes the callback upon EOF.

does_exist

Check whether a file/directory exists using a file-path.

graceful_copy

Do a copy but correctly handle short-writes and reads that might return a non- zero read count and EOF.

readseeker_to_readerat

A wrapper that allows an io.ReadSeeker to be used as a io.ReaderAt.

simplefileinfo

An implementation of os.FileInfo to support testing.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrSeekBeyondBound is returned when a seek is requested beyond the
	// statically-given file-size. No writes or seeks beyond boundaries are
	// supported with a statically-given file size.
	ErrSeekBeyondBound = errors.New("seek beyond boundary")
)

Functions

func CalculateSeek 

func CalculateSeek(currentOffset int64, delta int64, whence int, fileSize int64) (finalOffset int64, err error)

CalculateSeek calculates an offset in a file-stream given the parameters.

func CopyBytesBetweenPositions 

func CopyBytesBetweenPositions(rws io.ReadWriteSeeker, fromPosition, toPosition int64, count int) (n int, err error)

CopyBytesBetweenPositions will copy bytes from one position in the given RWS to an earlier position in the same RWS.

func DoesExist 

func DoesExist(filepath string) bool

DoesExist returns true if we can open the given file/path without error. We can't simply use `os.IsNotExist()` because we'll get a different error when the parent directory doesn't exist, and really the only important thing is if it exists *and* it's readable.

func GetOffset 

func GetOffset(s io.Seeker) int64

GetOffset returns the current offset of the Seeker and just panics if unable to find it.

func GracefulCopy 

func GracefulCopy(w io.Writer, r io.Reader, buffer []byte) (copyCount int, err error)

GracefulCopy willcopy while enduring lesser normal issues.

  • We'll ignore EOF if the read byte-count is more than zero. Only an EOF when zero bytes were read will terminate the loop.
  • Ignore short-writes. If less bytes were written than the bytes that were given, we'll keep trying until done.

func ListFiles 

func ListFiles(rootPath string, cb FileListFilterPredicate) (filesC chan VisitedFile, count int, errC chan error)

ListFiles feeds a continuous list of files from a recursive folder scan. An optional predicate can be provided in order to filter. When done, the `filesC` channel is closed. If there's an error, the `errC` channel will receive it.

func NewReadProgressWrapper 

func NewReadProgressWrapper(r io.Reader, progressCb ProgressFunc) io.Reader

NewReadProgressWrapper returns a new RPW instance.

func NewWriteProgressWrapper 

func NewWriteProgressWrapper(w io.Writer, progressCb ProgressFunc) io.Writer

NewWriteProgressWrapper returns a new WPW instance.

Types

type BouncebackReader 

type BouncebackReader struct {
	// contains filtered or unexported fields
}

BouncebackReader wraps a ReadSeeker, keeps track of our position, and seeks back to it before writing. This allows an underlying ReadWriteSeeker with an unstable position can still be used for a prolonged series of writes.

func NewBouncebackReader 

func NewBouncebackReader(rs io.ReadSeeker) (br *BouncebackReader, err error)

NewBouncebackReader returns a `*BouncebackReader` struct.

func (*BouncebackReader) Position 

func (br *BouncebackReader) Position() int64

Position returns the position that we're supposed to be at.

func (*BouncebackReader) Read 

func (br *BouncebackReader) Read(p []byte) (n int, err error)

Seek does a standard read.

func (*BouncebackReader) Seek 

func (br *BouncebackReader) Seek(offset int64, whence int) (newPosition int64, err error)

Seek does a seek to an arbitrary place in the `io.ReadSeeker`.

func (*BouncebackReader) StatsReads 

func (br *BouncebackReader) StatsReads() int

StatsReads returns the number of reads that have been attempted.

func (*BouncebackReader) StatsSeeks 

func (br *BouncebackReader) StatsSeeks() int

StatsSeeks returns the number of underlying seeks ("bounce-backs") that have been required.

type BouncebackWriter 

type BouncebackWriter struct {
	// contains filtered or unexported fields
}

BouncebackWriter wraps a WriteSeeker, keeps track of our position, and seeks back to it before writing. This allows an underlying ReadWriteSeeker with an unstable position can still be used for a prolonged series of writes.

func NewBouncebackWriter 

func NewBouncebackWriter(ws io.WriteSeeker) (bw *BouncebackWriter, err error)

NewBouncebackWriter returns a new `BouncebackWriter` struct.

func (*BouncebackWriter) Position 

func (bw *BouncebackWriter) Position() int64

Position returns the position that we're supposed to be at.

func (*BouncebackWriter) Seek 

func (bw *BouncebackWriter) Seek(offset int64, whence int) (newPosition int64, err error)

Seek puts us at a specific position in the internal writer for the next write/seek.

func (*BouncebackWriter) StatsSeeks 

func (bw *BouncebackWriter) StatsSeeks() int

StatsSeeks returns the number of seek operations.

func (*BouncebackWriter) StatsWrites 

func (bw *BouncebackWriter) StatsWrites() int

StatsWrites returns the number of write operations.

func (*BouncebackWriter) Write 

func (bw *BouncebackWriter) Write(p []byte) (n int, err error)

Write performs a write against the internal `WriteSeeker` starting at the position that we're supposed to be at.

type BoundedReadWriteSeekCloser 

type BoundedReadWriteSeekCloser struct {
	io.Closer
	*BoundedReadWriteSeeker
}

BoundedReadWriteSeekCloser wraps a RWS that is also a closer with boundaries. This proxies the RWS methods to the inner BRWS inside.

func NewBoundedReadWriteSeekCloser 

func NewBoundedReadWriteSeekCloser(rwsc ReadWriteSeekCloser, minimumOffset int64, staticFileSize int64) (brwsc *BoundedReadWriteSeekCloser, err error)

NewBoundedReadWriteSeekCloser returns a new BoundedReadWriteSeekCloser.

func (*BoundedReadWriteSeekCloser) Close 

func (rwsc *BoundedReadWriteSeekCloser) Close() (err error)

Close forwards calls to the inner RWS.

func (*BoundedReadWriteSeekCloser) Read 

func (rwsc *BoundedReadWriteSeekCloser) Read(buffer []byte) (readCount int, err error)

Read forwards calls to the inner RWS.

func (*BoundedReadWriteSeekCloser) Seek 

func (rwsc *BoundedReadWriteSeekCloser) Seek(offset int64, whence int) (newOffset int64, err error)

Seek forwards calls to the inner RWS.

func (*BoundedReadWriteSeekCloser) Write 

func (rwsc *BoundedReadWriteSeekCloser) Write(buffer []byte) (writtenCount int, err error)

Write forwards calls to the inner RWS.

type BoundedReadWriteSeeker 

type BoundedReadWriteSeeker struct {
	io.ReadWriteSeeker
	// contains filtered or unexported fields
}

BoundedReadWriteSeeker is a thin filter that ensures that no seeks can be done to offsets smaller than the one we were given. This supports libraries that might be expecting to read from the front of the stream being used on data that is in the middle of a stream instead.

func NewBoundedReadWriteSeeker 

func NewBoundedReadWriteSeeker(rws io.ReadWriteSeeker, minimumOffset int64, staticFileSize int64) (brws *BoundedReadWriteSeeker, err error)

NewBoundedReadWriteSeeker returns a new BoundedReadWriteSeeker instance.

func (*BoundedReadWriteSeeker) MinimumOffset 

func (brws *BoundedReadWriteSeeker) MinimumOffset() int64

MinimumOffset returns the configured minimum-offset.

func (*BoundedReadWriteSeeker) Read 

func (brws *BoundedReadWriteSeeker) Read(buffer []byte) (readCount int, err error)

Read forwards writes to the inner RWS.

func (*BoundedReadWriteSeeker) Seek 

func (brws *BoundedReadWriteSeeker) Seek(offset int64, whence int) (updatedOffset int64, err error)

Seek moves the offset to the given offset. Prevents offset from ever being moved left of `brws.minimumOffset`.

func (*BoundedReadWriteSeeker) Write 

func (brws *BoundedReadWriteSeeker) Write(buffer []byte) (writtenCount int, err error)

Write forwards writes to the inner RWS.

type FileListFilterPredicate 

type FileListFilterPredicate func(parent string, child os.FileInfo) (hit bool, err error)

FileListFilterPredicate is the callback predicate used for filtering.

type ProgressFunc 

type ProgressFunc func(n int, duration time.Duration, isEof bool) error

type ReadCounter 

type ReadCounter struct {
	// contains filtered or unexported fields
}

ReadCounter proxies read requests and maintains a counter of bytes read.

func NewReadCounter 

func NewReadCounter(r io.Reader) *ReadCounter

NewReadCounter returns a new `ReadCounter` struct wrapping a `Reader`.

func (*ReadCounter) Count 

func (rc *ReadCounter) Count() int

Count returns the total number of bytes read.

func (*ReadCounter) Read 

func (rc *ReadCounter) Read(b []byte) (n int, err error)

Read forwards a read to the underlying `Reader` while bumping the counter.

func (*ReadCounter) Reset 

func (rc *ReadCounter) Reset()

Reset resets the counter to zero.

type ReadProgressWrapper 

type ReadProgressWrapper struct {
	// contains filtered or unexported fields
}

ReadProgressWrapper wraps a reader and calls a callback after each read with count and duration info.

func (*ReadProgressWrapper) Read 

func (rpw *ReadProgressWrapper) Read(buffer []byte) (n int, err error)

Read reads data and calls the callback.

type ReadSeekerToReaderAt 

type ReadSeekerToReaderAt struct {
	// contains filtered or unexported fields
}

ReadSeekerToReaderAt is a wrapper that allows a ReadSeeker to masquerade as a ReaderAt.

func NewReadSeekerToReaderAt 

func NewReadSeekerToReaderAt(rs io.ReadSeeker) *ReadSeekerToReaderAt

NewReadSeekerToReaderAt returns a new ReadSeekerToReaderAt instance.

func (*ReadSeekerToReaderAt) ReadAt 

func (rstra *ReadSeekerToReaderAt) ReadAt(p []byte, offset int64) (n int, err error)

ReadAt is a wrapper that satisfies the ReaderAt interface.

Note that a requirement of ReadAt is that it doesn't have an effect on the offset in the underlying resource as well as that concurrent calls can be made to it. Since we're capturing the current offset in the underlying resource and then seeking back to it before returning, it is the responsibility of the caller to serialize (i.e. use a mutex with) these requests in order to eliminate race-conditions in the parallel-usage scenario.

Note also that, since ReadAt() is going to be called on a particular instance, that instance is going to internalize a file resource, that file- resource is provided by the OS, and [most] OSs are only gonna support one file-position per resource, locking is already going to be a necessary internal semantic of a ReaderAt implementation.

type ReadWriteSeekCloser 

type ReadWriteSeekCloser interface {
	io.ReadWriteSeeker
	io.Closer
}

ReadWriteSeekCloser satisfies `io.ReadWriteSeeker` and `io.Closer` interfaces.

func ReadWriteSeekNoopCloser 

func ReadWriteSeekNoopCloser(rws io.ReadWriteSeeker) ReadWriteSeekCloser

ReadWriteSeekNoopCloser wraps a `io.ReadWriteSeeker` with a no-op Close() call.

type SeekableBuffer 

type SeekableBuffer struct {
	// contains filtered or unexported fields
}

SeekableBuffer is a simple memory structure that satisfies `io.ReadWriteSeeker`.

func NewSeekableBuffer 

func NewSeekableBuffer() *SeekableBuffer

NewSeekableBuffer is a factory that returns a `*SeekableBuffer`.

func NewSeekableBufferWithBytes 

func NewSeekableBufferWithBytes(originalData []byte) *SeekableBuffer

NewSeekableBufferWithBytes is a factory that returns a `*SeekableBuffer`.

func (*SeekableBuffer) Bytes 

func (sb *SeekableBuffer) Bytes() []byte

Bytes returns the underlying slice.

func (*SeekableBuffer) Len 

func (sb *SeekableBuffer) Len() int

Len returns the number of bytes currently stored.

func (*SeekableBuffer) Read 

func (sb *SeekableBuffer) Read(p []byte) (n int, err error)

Read does a standard read against the internal slice.

func (*SeekableBuffer) Seek 

func (sb *SeekableBuffer) Seek(offset int64, whence int) (n int64, err error)

Seek does a standard seek on the internal slice.

func (*SeekableBuffer) Truncate 

func (sb *SeekableBuffer) Truncate(size int64) (err error)

Truncate either chops or extends the internal buffer.

func (*SeekableBuffer) Write 

func (sb *SeekableBuffer) Write(p []byte) (n int, err error)

Write does a standard write to the internal slice.

type SimpleFileInfo 

type SimpleFileInfo struct {
	// contains filtered or unexported fields
}

SimpleFileInfo is a simple `os.FileInfo` implementation useful for testing with the bare minimum.

func NewSimpleFileInfoWithDirectory 

func NewSimpleFileInfoWithDirectory(filename string, modTime time.Time) *SimpleFileInfo

NewSimpleFileInfoWithDirectory returns a new directory-specific SimpleFileInfo.

func NewSimpleFileInfoWithFile 

func NewSimpleFileInfoWithFile(filename string, size int64, mode os.FileMode, modTime time.Time) *SimpleFileInfo

NewSimpleFileInfoWithFile returns a new file-specific SimpleFileInfo.

func (*SimpleFileInfo) IsDir 

func (sfi *SimpleFileInfo) IsDir() bool

IsDir returns true if a directory.

func (*SimpleFileInfo) ModTime 

func (sfi *SimpleFileInfo) ModTime() time.Time

ModTime returns the modification time.

func (*SimpleFileInfo) Mode 

func (sfi *SimpleFileInfo) Mode() os.FileMode

Mode returns the file mode bits.

func (*SimpleFileInfo) Name 

func (sfi *SimpleFileInfo) Name() string

Name returns the base name of the file.

func (*SimpleFileInfo) Size 

func (sfi *SimpleFileInfo) Size() int64

Size returns the length in bytes for regular files; system-dependent for others.

func (*SimpleFileInfo) Sys 

func (sfi *SimpleFileInfo) Sys() interface{}

Sys returns internal state.

type VisitedFile 

type VisitedFile struct {
	Filepath string
	Info     os.FileInfo
	Index    int
}

VisitedFile is one visited file.

type WriteCounter 

type WriteCounter struct {
	// contains filtered or unexported fields
}

WriteCounter proxies write requests and maintains a counter of bytes written.

func NewWriteCounter 

func NewWriteCounter(w io.Writer) *WriteCounter

NewWriteCounter returns a new `WriteCounter` struct wrapping a `Writer`.

func (*WriteCounter) Count 

func (wc *WriteCounter) Count() int

Count returns the total number of bytes read.

func (*WriteCounter) Reset 

func (wc *WriteCounter) Reset()

Reset resets the counter to zero.

func (*WriteCounter) Write 

func (wc *WriteCounter) Write(b []byte) (n int, err error)

Write forwards a write to the underlying `Writer` while bumping the counter.

type WriteProgressWrapper 

type WriteProgressWrapper struct {
	// contains filtered or unexported fields
}

WriteProgressWrapper wraps a reader and calls a callback after each read with count and duration info.

func (*WriteProgressWrapper) Write 

func (wpw *WriteProgressWrapper) Write(buffer []byte) (n int, err error)

Write does a write and calls the callback.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.