fileio

Documentation

Index

• func CloseAndReport(f io.Closer, err *error)
• func FileSuffix(typ FileType) string
• func IsGrailRecordio(ft FileType) bool
• func MustClose(f io.Closer)
• type Closer
• type FileType
  • func DetermineType(filename string) FileType
• type ReadCloser
• type ReadWriteCloser
• type Reader
• type StorageAPI
  • func DetermineAPI(pathname string) StorageAPI
  • func SpellCorrectS3(s3path string) (StorageAPI, bool, string)
• type WriteCloser
• type Writer

Functions

func CloseAndReport

func CloseAndReport(f io.Closer, err *error)

CloseAndReport returns a defer-able helper that calls f.Close and reports errors, if any, to *err. Pass your function's named return error. Example usage:

func processFile(filename string) (_ int, err error) {
  f, err := os.Open(filename)
  if err != nil { ... }
  defer fileio.CloseAndReport(f, &err)
  ...
}

If your function returns with an error, any f.Close error will be chained appropriately.

Deprecated: Use errors.CleanUp directly.

func FileSuffix

func FileSuffix(typ FileType) string

FileSuffix returns the filename suffix associated with the specified FileType.

func IsGrailRecordio

func IsGrailRecordio(ft FileType) bool

IsGrailRecordio returns true if the filetype is one of the Grail recordio types.

func MustClose

func MustClose(f io.Closer)

MustClose is a defer-able function that calls f.Close and panics on error.

Example:

f, err := os.Open(filename)
if err != nil { panic(err) }
defer fileio.MustClose(f)
...

Types

type Closer

type Closer interface {
	io.Closer
	Name() string
}

Closer is io.Closer with an additional Name method that returns the name of the original source of the closer.

type FileType

type FileType int

FileType represents the type of a file based on its filename

const (
	// Other represents a filetype other than the ones supported here.
	Other FileType = iota
	// Gzip file.
	Gzip
	// Bzip2 file.
	Bzip2
	// GrailRIO recordio.
	GrailRIO
	// GrailRIOPacked packed recordio.
	GrailRIOPacked
	// GrailRIOPackedEncrypted encrypted packed recordio.
	GrailRIOPackedEncrypted
	// GrailRIOPackedCompressed compressed packed recordio.
	GrailRIOPackedCompressed
	// GrailRIOPackedCompressedAndEncrypted compressed and encrypted packed recordio.
	GrailRIOPackedCompressedAndEncrypted
	// JSON text file
	JSON
	// Zstd format.
	// https://facebook.github.io/zstd/
	// https://tools.ietf.org/html/rfc8478
	Zstd
)

func DetermineType

func DetermineType(filename string) FileType

DetermineType determines the type of the file given its filename.

type ReadCloser

type ReadCloser interface {
	io.ReadCloser
	Name() string
}

ReadCloser is io.ReadCloser with an additional Name method that returns the name of the original source of the reader.

type ReadWriteCloser

type ReadWriteCloser interface {
	io.ReadWriteCloser
	Name() string
}

ReadWriteCloser is an interface that implements io.ReadWriteCloser with an additional Name method that returns the name of the original source of the writer.

type Reader

type Reader interface {
	io.Reader
	Name() string
}

Reader is io.Reader with an additional Name method that returns the name of the original source of the reader.

type StorageAPI

type StorageAPI int

StorageAPI represents the Storage API required to access a file.

const (
	// LocalAPI represents a local fileystem accessible via a unix/posix API
	// and hence the io/os packages.
	LocalAPI StorageAPI = iota
	// S3API represents an Amazon S3 API.
	S3API
)

func DetermineAPI

func DetermineAPI(pathname string) StorageAPI

DetermineAPI determines the Storage API that stores the file referred to by pathname.

func SpellCorrectS3

func SpellCorrectS3(s3path string) (StorageAPI, bool, string)

SpellCorrectS3 returns true if the S3 path looks like an S3 path and returns the spell corrected path. That is, it returns true for common mispellings such as those show below along with the corrected s3://<path> s3:///<path> s3:<path> s3:/<path> s://<path> s:/<path> s3//<path>

type WriteCloser

type WriteCloser interface {
	io.WriteCloser
	Name() string
}

WriteCloser is io.WriteCloser with an additional Name method that returns the name of the original source of the writer.

type Writer

type Writer interface {
	io.Writer
	Name() string
}

Writer is io.Writer with an additional Name method that returns the name of the original source of the writer.