compress

package
v0.0.0-...-1dc401f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 27, 2024 License: MIT Imports: 28 Imported by: 0

Documentation

Overview

Package compress provides wrappers for Fs and Object which implement compression.

Index

Constants

View Source 
const (
	Uncompressed = 0
	Gzip         = 2
)

Compression modes

Variables

This section is empty.

Functions

func NewFs 

func NewFs(ctx context.Context, name, rpath string, m configmap.Mapper) (fs.Fs, error)

NewFs constructs an Fs from the path, container:path

Types

type Fs 

type Fs struct {
	fs.Fs
	// contains filtered or unexported fields
}

Fs represents a wrapped fs.Fs

func (*Fs) About 

func (f *Fs) About(ctx context.Context) (*fs.Usage, error)

About gets quota information from the Fs

func (*Fs) ChangeNotify 

func (f *Fs) ChangeNotify(ctx context.Context, notifyFunc func(string, fs.EntryType), pollIntervalChan <-chan time.Duration)

ChangeNotify calls the passed function with a path that has had changes. If the implementation uses polling, it should adhere to the given interval.

func (*Fs) CleanUp 

func (f *Fs) CleanUp(ctx context.Context) error

CleanUp the trash in the Fs

Implement this if you have a way of emptying the trash or otherwise cleaning up old versions of files.

func (*Fs) Copy 

func (f *Fs) Copy(ctx context.Context, src fs.Object, remote string) (fs.Object, error)

Copy src to this remote using server side copy operations.

This is stored with the remote path given.

It returns the destination Object and a possible error.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantCopy

func (*Fs) DirCacheFlush 

func (f *Fs) DirCacheFlush()

DirCacheFlush resets the directory cache - used in testing as an optional interface

func (*Fs) DirMove 

func (f *Fs) DirMove(ctx context.Context, src fs.Fs, srcRemote, dstRemote string) error

DirMove moves src, srcRemote to this remote at dstRemote using server side move operations.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantDirMove

If destination exists then return fs.ErrorDirExists

func (*Fs) DirSetModTime 

func (f *Fs) DirSetModTime(ctx context.Context, dir string, modTime time.Time) error

DirSetModTime sets the directory modtime for dir

func (*Fs) Features 

func (f *Fs) Features() *fs.Features

Features returns the optional features of this Fs

func (*Fs) Hashes 

func (f *Fs) Hashes() hash.Set

Hashes returns the supported hash sets.

func (*Fs) List 

func (f *Fs) List(ctx context.Context, dir string) (entries fs.DirEntries, err error)

List the objects and directories in dir into entries. The entries can be returned in any order but should be for a complete directory.

dir should be "" to list the root, and should not have trailing slashes.

This should return ErrDirNotFound if the directory isn't found. List entries and process them

func (*Fs) ListR 

func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) (err error)

ListR lists the objects and directories of the Fs starting from dir recursively into out.

dir should be "" to start from the root, and should not have trailing slashes.

This should return ErrDirNotFound if the directory isn't found.

It should call callback for each tranche of entries read. These need not be returned in any particular order. If callback returns an error then the listing will stop immediately.

Don't implement this unless you have a more efficient way of listing recursively that doing a directory traversal.

func (*Fs) MergeDirs 

func (f *Fs) MergeDirs(ctx context.Context, dirs []fs.Directory) error

MergeDirs merges the contents of all the directories passed in into the first one and rmdirs the other directories.

func (*Fs) Mkdir 

func (f *Fs) Mkdir(ctx context.Context, dir string) error

Mkdir makes the directory (container, bucket)

Shouldn't return an error if it already exists

func (*Fs) MkdirMetadata 

func (f *Fs) MkdirMetadata(ctx context.Context, dir string, metadata fs.Metadata) (fs.Directory, error)

MkdirMetadata makes the root directory of the Fs object

func (*Fs) Move 

func (f *Fs) Move(ctx context.Context, src fs.Object, remote string) (fs.Object, error)

Move src to this remote using server side move operations.

This is stored with the remote path given.

It returns the destination Object and a possible error.

Will only be called if src.Fs().Name() == f.Name()

If it isn't possible then return fs.ErrorCantMove

func (*Fs) Name 

func (f *Fs) Name() string

Name of the remote (as passed into NewFs)

func (*Fs) NewObject 

func (f *Fs) NewObject(ctx context.Context, remote string) (fs.Object, error)

NewObject finds the Object at remote.

func (*Fs) Precision 

func (f *Fs) Precision() time.Duration

Precision returns the precision of this Fs 

func (f *Fs) PublicLink(ctx context.Context, remote string, duration fs.Duration, unlink bool) (string, error)

PublicLink generates a public link to the remote path (usually readable by anyone)

func (*Fs) Purge 

func (f *Fs) Purge(ctx context.Context, dir string) error

Purge all files in the root and the root directory

Implement this if you have a way of deleting all the files quicker than just running Remove() on the result of List()

Return an error if it doesn't exist

func (*Fs) Put 

func (f *Fs) Put(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

Put in to the remote path with the modTime given of the given size

May create the object even if it returns an error - if so will return the object and the error, otherwise will return nil and the error

func (*Fs) PutStream 

func (f *Fs) PutStream(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (fs.Object, error)

PutStream uploads to the remote path with the modTime given of indeterminate size

func (*Fs) Rmdir 

func (f *Fs) Rmdir(ctx context.Context, dir string) error

Rmdir removes the directory (container, bucket) if empty

Return an error if it doesn't exist or isn't empty

func (*Fs) Root 

func (f *Fs) Root() string

Root of the remote (as passed into NewFs)

func (*Fs) SetWrapper 

func (f *Fs) SetWrapper(wrapper fs.Fs)

SetWrapper sets the Fs that is wrapping this Fs

func (*Fs) Shutdown 

func (f *Fs) Shutdown(ctx context.Context) error

Shutdown the backend, closing any background tasks and any cached connections.

func (*Fs) String 

func (f *Fs) String() string

Return a string version

func (*Fs) UnWrap 

func (f *Fs) UnWrap() fs.Fs

UnWrap returns the Fs that this Fs is wrapping

func (*Fs) WrapFs 

func (f *Fs) WrapFs() fs.Fs

WrapFs returns the Fs that is wrapping this Fs

type Object 

type Object struct {
	fs.Object // Wraps around data object for this object
	// contains filtered or unexported fields
}

Object with external metadata

func (*Object) Fs 

func (o *Object) Fs() fs.Info

Fs returns read only access to the Fs that this object is part of

func (*Object) GetTier 

func (o *Object) GetTier() string

GetTier returns storage tier or class of the Object

func (*Object) Hash 

func (o *Object) Hash(ctx context.Context, ht hash.Type) (string, error)

Hash returns the selected checksum of the file If no checksum is available it returns ""

func (*Object) ID 

func (o *Object) ID() string

ID returns the ID of the Object if known, or "" if not

func (*Object) Metadata 

func (o *Object) Metadata(ctx context.Context) (fs.Metadata, error)

Metadata returns metadata for an object

It should return nil if there is no Metadata

func (*Object) MimeType 

func (o *Object) MimeType(ctx context.Context) string

MimeType returns the MIME type of the file

func (*Object) Open 

func (o *Object) Open(ctx context.Context, options ...fs.OpenOption) (rc io.ReadCloser, err error)

Open opens the file for read. Call Close() on the returned io.ReadCloser. Note that this call requires quite a bit of overhead.

func (*Object) Remote 

func (o *Object) Remote() string

Remote returns the remote path

func (*Object) Remove 

func (o *Object) Remove(ctx context.Context) error

Remove removes this object

func (*Object) SetMetadata 

func (o *Object) SetMetadata(ctx context.Context, metadata fs.Metadata) error

SetMetadata sets metadata for an Object

It should return fs.ErrorNotImplemented if it can't set metadata

func (*Object) SetTier 

func (o *Object) SetTier(tier string) error

SetTier performs changing storage tier of the Object if multiple storage classes supported

func (*Object) Size 

func (o *Object) Size() int64

Size returns the size of the file

func (*Object) String 

func (o *Object) String() string

Return a string version

func (*Object) UnWrap 

func (o *Object) UnWrap() fs.Object

UnWrap returns the wrapped Object

func (*Object) Update 

func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (err error)

Update in to the object with the modTime given of the given size

type ObjectInfo 

type ObjectInfo struct {
	// contains filtered or unexported fields
}

ObjectInfo describes a wrapped fs.ObjectInfo for being the source

func (*ObjectInfo) Fs 

func (o *ObjectInfo) Fs() fs.Info

Fs returns read only access to the Fs that this object is part of

func (*ObjectInfo) GetTier 

func (o *ObjectInfo) GetTier() string

GetTier returns storage tier or class of the Object

func (*ObjectInfo) Hash 

func (o *ObjectInfo) Hash(ctx context.Context, ht hash.Type) (string, error)

Hash returns the selected checksum of the file If no checksum is available it returns ""

func (*ObjectInfo) ID 

func (o *ObjectInfo) ID() string

ID returns the ID of the Object if known, or "" if not

func (*ObjectInfo) Metadata 

func (o *ObjectInfo) Metadata(ctx context.Context) (fs.Metadata, error)

Metadata returns metadata for an object

It should return nil if there is no Metadata

func (*ObjectInfo) MimeType 

func (o *ObjectInfo) MimeType(ctx context.Context) string

MimeType returns the content type of the Object if known, or "" if not

func (*ObjectInfo) ModTime 

func (o *ObjectInfo) ModTime(ctx context.Context) time.Time

ModTime returns the modification time

func (*ObjectInfo) Remote 

func (o *ObjectInfo) Remote() string

Remote returns the remote path

func (*ObjectInfo) Size 

func (o *ObjectInfo) Size() int64

Size returns the size of the file

func (*ObjectInfo) Storable 

func (o *ObjectInfo) Storable() bool

Storable returns whether object is storable

func (*ObjectInfo) String 

func (o *ObjectInfo) String() string

String returns string representation

func (*ObjectInfo) UnWrap 

func (o *ObjectInfo) UnWrap() fs.Object

UnWrap returns the Object that this Object is wrapping or nil if it isn't wrapping anything

type ObjectMetadata 

type ObjectMetadata struct {
	Mode                int    // Compression mode of the file.
	Size                int64  // Size of the object.
	MD5                 string // MD5 hash of the file.
	MimeType            string // Mime type of the file
	CompressionMetadata sgzip.GzipMetadata
}

ObjectMetadata describes the metadata for an Object.

type Options 

type Options struct {
	Remote           string        `config:"remote"`
	CompressionMode  string        `config:"mode"`
	CompressionLevel int           `config:"level"`
	RAMCacheLimit    fs.SizeSuffix `config:"ram_cache_limit"`
}

Options defines the configuration for this backend

type ReadCloserWrapper 

type ReadCloserWrapper struct {
	io.Reader
	io.Closer
}

ReadCloserWrapper combines a Reader and a Closer to a ReadCloser

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.