mem

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 1, 2023 License: Apache-2.0 Imports: 4 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetMemFileFs

func GetMemFileFs() afero.Fs

GetMemFileFs - returns the current memory file-system being used by ParquetFile

func NewMemFileWriter

func NewMemFileWriter(name string, f OnCloseFunc) (source.ParquetFile, error)

NewMemFileWriter - intiates and creates an instance of MemFiles NOTE: there is no NewMemFileReader as this particular type was written to handle in-memory conversions and offloading. The results of conversion can then be stored and read via HDFS, LocalFS, etc without the need for loading the file back into memory directly

func SetInMemFileFs

func SetInMemFileFs(fs *afero.Fs)

SetInMemFileFs - overrides local in-memory fileSystem NOTE: this is set by NewMemFileWriter is created and memFs is still nil

Types

type MemFile

type MemFile struct {
	FilePath string
	File     afero.File
	OnClose  OnCloseFunc
}

MemFile - ParquetFile type for in-memory file operations

func (*MemFile) Close

func (fs *MemFile) Close() error

Close - close file and execute OnCloseFunc

func (*MemFile) Create

func (fs *MemFile) Create(name string) (source.ParquetFile, error)

Create - create in-memory file

func (*MemFile) Open

func (fs *MemFile) Open(name string) (source.ParquetFile, error)

Open - open file in-memory

func (*MemFile) Read

func (fs *MemFile) Read(b []byte) (cnt int, err error)

Read - read file

func (*MemFile) Seek

func (fs *MemFile) Seek(offset int64, pos int) (int64, error)

Seek - seek function

func (*MemFile) Write

func (fs *MemFile) Write(b []byte) (n int, err error)

Write - write file in-memory

type OnCloseFunc

type OnCloseFunc func(string, io.Reader) error

OnCloseFunc function type, handles what to do after converted file is closed in-memory. Close() will pass the filename string and data as io.reader

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL