reflux

README

reflux

reflux is a Go package designed to manage file transfers with an emphasis on resuming interrupted transfers. Its main features include the maintenance of comprehensive metadata and additional data associated with each file. By leveraging BoltDB, a lightweight embedded key-value store, Reflux ensures persistent storage of all file-related information.

The utility of Reflux extends to server information storage, allowing developers to store and retrieve server details such as address, port, and user. This makes Reflux a versatile tool for developers looking to manage file transfers and server information in their applications.

Whether you're building a client-server application, a peer-to-peer network, or simply handling files in a local setting, Reflux provides a set of utilities to make your work easier and more efficient. From storing file metadata, tracking transfer status, and handling additional data associated with files, Reflux has got you covered.

Features

• File transfer management: The TransferManager struct allows you to manage file transfers, including storing file metadata, tracking transfer status, and handling additional data associated with files.
• Server information storage: The package provides functionality to store and retrieve server information, such as server address, port, and user.
• Database persistence: The file metadata, additional data, and server information are stored persistently using BoltDB, a lightweight embedded key-value store.

Installation

To install the reflux package, use the following command:

go get gopkg.in/ro-ag/reflux.v0

Usage

Importing the package

To use the reflux package in your Go code, import it as follows:

import "gopkg.in/ro-ag/reflux.v0"

Creating a new transfer manager

To create a new transfer manager, use the NewTransferManager function:

tm, err := reflux.NewTransferManager()
if err != nil {
    // Handle error
}
defer tm.Finish()

This initializes the TransferManager, opens the database, and sets up the necessary resources. Make sure to defer the Finish method to perform cleanup operations when you're done using the TransferManager.

Storing and retrieving file metadata

To store file metadata, use the StoreOrUpdate method of the FileMetadataMap interface:

metadata := reflux.FileMetadata{
    SourcePath:       "/path/to/source/file.txt",
    TargetPath:       "/path/to/target/file.txt",
    Status:           reflux.StatusNotStarted,
    BytesTransferred: 0,
    // ... additional fields
}

err := tm.Files.StoreOrUpdate(metadata)
if err != nil {
    // Handle error
}

To retrieve file metadata, use the Load method of the FileMetadataMap interface:

fileNameKey := "/path/to/source/file.txt"
metadata, found := tm.Files.Load(fileNameKey) // this is not actually a file name, but a key
if found {
    // File metadata found
    fmt.Println(metadata)
} else {
    // File metadata not found
}

Storing and retrieving server information

To store server information, use the StoreOrUpdateServerInfo method of the TransferManager:

info := reflux.CreateServerInfo("example.com", 8080, "user")

err := tm.StoreOrUpdateServerInfo(info)
if err != nil {
    // Handle error
}

To retrieve server information, use the GetServerInfo method of the TransferManager:

serverInfo, err := tm.GetServerInfo()
if err != nil {
    // Handle error
} else {
    fmt.Println("Server Address:", serverInfo.Address)
    fmt.Println("Server Port:", serverInfo.Port)
    fmt.Println("Server User:", serverInfo.User)
}

Additional functionality

The reflux package provides additional functionality for managing file transfers, including updating transfer status, starting transfers, setting errors, and more. Refer to the package documentation and the source code for detailed usage examples and available methods.

Documentation

Index

• Variables
• type AttributesMap
• type FileMetadata
• type FileMetadataMap
• type ServerInfo
  • func CreateServerInfo(address string, port int, user string) (*ServerInfo, error)
• type Transfer
• type TransferManager
  • func NewTransferManager() (*TransferManager, error)
  • func (tm *TransferManager) Close() error
  • func (tm *TransferManager) Finish() error
  • func (tm *TransferManager) GetServerInfo() (*ServerInfo, error)
  • func (tm *TransferManager) IsPreexisting() bool
  • func (tm *TransferManager) StoreOrUpdateServerInfo(info *ServerInfo) error
• type TransferStatus
  • func (i TransferStatus) String() string

Variables

var (
	ErrServerInfoNotSet     = errors.New("server info not set")
	ErrInvalidAddressFormat = errors.New("invalid address format")
)

var ErrAttBucketNotFound = errors.Errorf("bucket '%s' not found", additionalDataBucket)

Types

type AttributesMap

type AttributesMap interface {

	// GetSlice returns a slice of additional data
	GetSlice() ([]any, error)

	// StoreOrUpdate stores or updates the additional data in the database.
	StoreOrUpdate(key string, data any) error

	// Load returns the additional data for the given key.
	Load(key string) (any, bool)

	// Delete deletes the additional data for the given key.
	Delete(key string) error

	// Exists returns true if the additional data for the given key exists.
	Exists(key string) bool
	// contains filtered or unexported methods
}

type FileMetadata

type FileMetadata struct {
	SourcePath       string         // The path of the file on the local machine
	TargetPath       string         // The path of the file on the remote machine
	Status           TransferStatus // The status of the transfer
	BytesTransferred int            // The number of bytes transferred
	TimeStart        time.Time      // The time the transfer started
	TimeEnd          time.Time      // The time the transfer ended
	ErrorMsg         string         // The error that occurred during the transfer
}

FileMetadata contains information about a file transfer.

type FileMetadataMap

type FileMetadataMap interface {

	// StoreOrUpdate stores or updates the file metadata in the database.
	// It encodes the file metadata and stores it in the Lock File (BoltDB database).
	StoreOrUpdate(metadata FileMetadata) error

	// Load returns the file metadata for the given source path.
	Load(sourcePath string) (FileMetadata, bool)

	// Delete deletes the file metadata for the given source path.
	Delete(sourcePath string) error

	// Operate operates on the file metadata for the given source path.
	Operate(op Transfer) ([]FileMetadata, error)

	// GetSlice returns a slice of file metadata
	GetSlice() ([]FileMetadata, error)

	// UpdateStatus updates the status of the file metadata for the given source path.
	UpdateStatus(sourcePath string, status TransferStatus, bytesTransferred int, err error) error

	// Start starts the transfer for the given source path.
	Start(sourcePath string) error

	// SetError sets the error for the given source path.
	SetError(sourcePath string, err error) error

	// SetSuccess sets the success for the given source path.
	SetSuccess(sourcePath string, bytesTransferred int) error
	// contains filtered or unexported methods
}

FileMetadataMap provides a synchronized map for storing and managing file metadata.

type ServerInfo

type ServerInfo struct {
	Address string // The address of the server
	Port    int    // The port of the server
	User    string // The user of the server
}

ServerInfo represents information about the server.

func CreateServerInfo

func CreateServerInfo(address string, port int, user string) (*ServerInfo, error)

CreateServerInfo creates a new instance of the ServerInfo interface.

type Transfer

type Transfer func(sourcePath string, targetPath string) (int, error)

type TransferManager

type TransferManager struct {
	Files      FileMetadataMap // type FileMetadata, to avoid race conditions Key is the file path
	Attributes AttributesMap   // Developers can use this to store additional data, for example command flags the developer is using to run the command
	// contains filtered or unexported fields
}

TransferManager manages file transfers and server information.

func NewTransferManager

func NewTransferManager() (*TransferManager, error)

NewTransferManager creates a new TransferManager instance. It initializes the lock file path, opens the database, and initializes the buckets. If the lock file already exists, it loads the existing data from the database.

func (*TransferManager) Close

func (tm *TransferManager) Close() error

Close closes the TransferManager and performs cleanup operations. It syncs the database, closes the database connection, and removes the lock file.

func (*TransferManager) Finish

func (tm *TransferManager) Finish() error

func (*TransferManager) GetServerInfo

func (tm *TransferManager) GetServerInfo() (*ServerInfo, error)

GetServerInfo returns the server information. If the server information is not set, it returns nil and the ErrServerInfoNotSet error.

func (*TransferManager) IsPreexisting

func (tm *TransferManager) IsPreexisting() bool

IsPreexisting returns whether the lock file already existed. this is useful to get the latest run status and resume the transfer.

func (*TransferManager) StoreOrUpdateServerInfo

func (tm *TransferManager) StoreOrUpdateServerInfo(info *ServerInfo) error

StoreOrUpdateServerInfo stores the server information in the database. It encodes the server info and stores it in the Lock File (BoltDB database).

type TransferStatus

type TransferStatus uint8

TransferStatus represents the status of a file transfer.

const (
	StatusNotStarted TransferStatus = iota
	StatusInProgress
	StatusCompleted
	StatusFailed
)

func (TransferStatus) String

func (i TransferStatus) String() string