common

package
v0.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 1, 2024 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Index

Examples

Constants

View Source 
const (
	ErrFileNotFound     = "file not found"
	ErrAccessDenied     = "access denied"
	ErrInvalidConfig    = "invalid backend configuration"
	ErrOperationTimeout = "operation timeout"
)

Common error definitions.

Variables

View Source 
var CreateDummyBackend func(cfg *Config) StorageBackend = nil

CreateDummyBackend function that returns a pre-initialized DummyBackend.

Functions

This section is empty.

Types

type B2Backend 

type B2Backend struct {
	s3iface.S3API
	// contains filtered or unexported fields
}

B2Backend is struct that holds active B2 session.

func CreateB2Backend 

func CreateB2Backend(cfg *Config) *B2Backend

CreateB2Backend is the B2Backend factory function.

func (*B2Backend) GetFileInfo 

func (b2 *B2Backend) GetFileInfo(uri *url.URL) (*FileInfo, error)

GetFileInfo returns a FileInfo struct filled with information about object defined by the input URI. Input URI must follow the pattern: b2://bucket/path/to/key.

func (*B2Backend) ListFiles 

func (b2 *B2Backend) ListFiles(uri *url.URL) ([]FileInfo, error)

ListFiles return an array of FileInfo structs filled with information about objects defined by the input URI. Input URI must follow the pattern: b2://bucket/path/to/prefix.

func (*B2Backend) RemoveFile added in v0.2.0 

func (b2 *B2Backend) RemoveFile(uri *url.URL) error

RemoveFile removes an object under the given URI. Object URI must follow the pattern: b2://bucket/path/to/key.

func (*B2Backend) StoreFile 

func (b2 *B2Backend) StoreFile(inputStream io.ReaderAt, contentLength int64, uri *url.URL) error

StoreFile writes data from `input` to output URI. Output URI must follow the pattern: b2://bucket/path/to/key.

type Config 

type Config struct {
	S3 struct {
		Region string `yaml:"region" env:"SQUIRRELUP_S3_REGION,overwrite" default:""`
		ID     string `yaml:"id" env:"SQUIRRELUP_S3_ID,overwrite" default:""`
		Secret string `yaml:"secret" env:"SQUIRRELUP_S3_SECRET,overwrite" default:""`
		Token  string `yaml:"token" env:"SQUIRRELUP_S3_TOKEN,overwrite" default:""`
	} `yaml:"s3"`
	Encryption struct {
		Pubkey string `yaml:"pubkey" env:"SQUIRRELUP_PUBKEY,overwrite" default:""`
	} `yaml:"encryption"`
	Backup struct {
		Hours float64 `yaml:"hours" env:"SQUIRRELUP_BACKUP_HOURS,overwrite" default:"240"`
		Name  string  `yaml:"name" env:"SQUIRRELUP_BACKUP_FILENAME,overwrite" default:"2006-01-02T15-0700"`
	} `yaml:"backup"`
	Internal struct {
		Reporter ProgressReporter
	}
}

Config struct contains configurations for SQUIRRELUP. Currently it contains four sections:

  • S3 configuration
  • Encryption configuration
  • Backup configuration
  • Internal configuration

func (*Config) LoadConfigFromEnv added in v0.2.0 

func (cfg *Config) LoadConfigFromEnv() error

LoadConfigFromEnv loads configuration into a `Config` struct from environment variables.

func (*Config) LoadConfigFromFile added in v0.2.0 

func (cfg *Config) LoadConfigFromFile(file io.Reader) error

LoadConfigFromFile loads configuration into a `Config` struct from YAML file.

func (*Config) SetDefaultValues added in v0.2.0 

func (cfg *Config) SetDefaultValues() error

SetDefaultValues reset Config fields to default values. An implementation handling a generic struct: 

func setDefaultValues() error {
	if reflect.TypeOf(ptr).Kind() != reflect.Ptr {
		return fmt.Errorf("Not a pointer")
	}
	valueof := reflect.ValueOf(ptr).Elem()
	if ! valueof.IsValid() {
		return fmt.Errorf("setDefaultValues called on a nil pointer")
	}
	fmt.Printf("setDefaultValues: kind = %v\n", valueof.Kind())
	if valueof.Kind() == reflect.Struct {
		if err := setDefaultValuesStruct(valueof); err != nil {
			// fmt.Printf("setDefaultValues: err = %v\n", err)
			return err
		}
		return nil
	}
	return fmt.Errorf("setDefaultValues called with unsupported value of kind '%v'", valueof.Kind())
}

type DummyBackend 

type DummyBackend struct {
	// contains filtered or unexported fields
}

DummyBackend defines a dummy backend.

func (*DummyBackend) GenerateDummyFiles added in v0.3.0 

func (d *DummyBackend) GenerateDummyFiles(path string, number uint64)

GenerateDummyFiles generate dummy file info list.

func (*DummyBackend) GetDummyError added in v0.3.0 

func (d *DummyBackend) GetDummyError() error

GetDummyError get dummy error.

func (*DummyBackend) GetDummyFiles added in v0.3.0 

func (d *DummyBackend) GetDummyFiles() []FileInfo

GetDummyFiles get dummy file info list.

func (*DummyBackend) GetFileInfo 

func (d *DummyBackend) GetFileInfo(uri *url.URL) (*FileInfo, error)

GetFileInfo returns a FileInfo struct filled with information about object defined by the input URI. Input URI must follow the pattern: dummy://path/to/file.

func (*DummyBackend) ListFiles 

func (d *DummyBackend) ListFiles(uri *url.URL) ([]FileInfo, error)

ListFiles return an array of FileInfo structs filled with information about objects defined by the input URI. Input URI must follow the pattern: dummy://path/to/dir.

func (*DummyBackend) RemoveFile added in v0.2.0 

func (d *DummyBackend) RemoveFile(uri *url.URL) error

RemoveFile remove objects defined by the input URI. Input URI must follow the pattern: dummy://path/to/dir.

func (*DummyBackend) SetDummyError added in v0.3.0 

func (d *DummyBackend) SetDummyError(err error)

SetDummyError set dummy error.

func (*DummyBackend) StoreFile 

func (d *DummyBackend) StoreFile(input io.ReaderAt, length int64, uri *url.URL) error

StoreFile writes a data from `input` to output URI. Output URI must follow the pattern: dummy://path/to/file.

type DummyProgressReporter added in v0.3.2 

type DummyProgressReporter struct {
}

DummyProgressReporter is just a stub.

func (*DummyProgressReporter) AdvanceTask added in v0.3.2 

func (dummy *DummyProgressReporter) AdvanceTask(index int, increment int64) error

AdvanceTask stub.

func (*DummyProgressReporter) CreateFileTask added in v0.3.2 

func (dummy *DummyProgressReporter) CreateFileTask(size int64) (int, error)

CreateFileTask stub.

func (*DummyProgressReporter) DescribeTask added in v0.3.2 

func (dummy *DummyProgressReporter) DescribeTask(index int, description string) error

DescribeTask stub.

func (*DummyProgressReporter) FinishTask added in v0.3.2 

func (dummy *DummyProgressReporter) FinishTask(index int) error

FinishTask stub.

type FileInfo 

type FileInfo struct {
	// contains filtered or unexported fields
}

FileInfo contains file information.

func (*FileInfo) IsFile added in v0.3.0 

func (fi *FileInfo) IsFile() bool

IsFile returns true if this object represents a file.

func (*FileInfo) Modified added in v0.2.0 

func (fi *FileInfo) Modified() time.Time

Modified returns last modified date of the file object.

func (*FileInfo) Name added in v0.2.0 

func (fi *FileInfo) Name() string

Name returns name of the file object.

func (*FileInfo) Size added in v0.2.0 

func (fi *FileInfo) Size() uint64

Size returns size of the file object.

type MultiProgressbarReporter added in v0.3.2 

type MultiProgressbarReporter struct {
	// contains filtered or unexported fields
}

MultiProgressbarReporter reportes progress of individual taks via progressbars.

Example

test cases for MultiProgressbarReporter FileTask 

var index int

// Setup Test
mockMPR := NewMultiProgressbarReporter(os.Stdout)
index, _ = mockMPR.CreateFileTask(100)
_ = mockMPR.DescribeTask(index, "test")

// Simulate some work
time.Sleep(1 * time.Second)
_ = mockMPR.AdvanceTask(index, 10)

Output:

test  10% |████                                    | (10 B/s)
Example (Spinner) 
var index int

// Setup Test
mockMPR := NewMultiProgressbarReporter(os.Stdout)
index, _ = mockMPR.CreateFileTask(-1)
_ = mockMPR.DescribeTask(index, "test")

// Simulate some work
time.Sleep(1 * time.Second)
_ = mockMPR.AdvanceTask(index, 10)

Output:

- test (10 B/s) [1s]

func NewMultiProgressbarReporter added in v0.3.2 

func NewMultiProgressbarReporter(output io.Writer) *MultiProgressbarReporter

NewMultiProgressbarReporter creates a MultiProgressBarReporter with a given `output`.

func (*MultiProgressbarReporter) AdvanceTask added in v0.3.2 

func (mpr *MultiProgressbarReporter) AdvanceTask(index int, increment int64) error

AdvanceTask advances the progress by `increment` on a task specified by `index`.

func (*MultiProgressbarReporter) CreateFileTask added in v0.3.2 

func (mpr *MultiProgressbarReporter) CreateFileTask(size int64) (int, error)

CreateFileTask will create an invisible progressbar for a file task. It will be displayed upon first call to AdvanceTask.

func (*MultiProgressbarReporter) DescribeTask added in v0.3.2 

func (mpr *MultiProgressbarReporter) DescribeTask(index int, description string) error

DescribeTask set desription of a task specified by `index`.

func (*MultiProgressbarReporter) FinishTask added in v0.3.2 

func (mpr *MultiProgressbarReporter) FinishTask(index int) error

FinishTask finish task specified by `index`.

type ProgressReporter added in v0.3.2 

type ProgressReporter interface {
	AdvanceTask(int, int64) error
	CreateFileTask(int64) (int, error)
	DescribeTask(int, string) error
	FinishTask(int) error
}

ProgressReporter is a generic interface to setting up progress reporting. Currently, it provisions following methods:

  • AdvanceTask advance progress on a task given by its index by a cetain amount.
  • CreateFileTask creates a new file task and return its index.
  • DescribeTask change task description given by its index.
  • FinishTask finish a task given by its index.

type StorageBackend 

type StorageBackend interface {
	GetFileInfo(*url.URL) (*FileInfo, error)
	ListFiles(*url.URL) ([]FileInfo, error)
	StoreFile(io.ReaderAt, int64, *url.URL) error
	RemoveFile(*url.URL) error
}

StorageBackend is a generic interface to storage backends. Currently, it provisions following methods:

  • GetFileInfo to get file information in FileInfo struct.
  • ListFiles to list files under a given URI.
  • StoreFile to store data to a given URI.
  • RemoveFile to remove files under a given URI.

func CreateStorageBackend 

func CreateStorageBackend(uri *url.URL, cfg *Config) (StorageBackend, error)

CreateStorageBackend is a StorageBackend factory function.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.