upload_utils

package
v1.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 29, 2024 License: Apache-2.0 Imports: 32 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FindMinMissingInteger added in v1.1.0 

func FindMinMissingInteger(arr []int) int

FindMinMissingInteger finds the minimum missing integer in a sorted array

func NewUploadStatusMonitor 

func NewUploadStatusMonitor(uploadStatusMap map[string]*FileInfo, orderedFileList *[]string, hidden bool) tea.Model

NewUploadStatusMonitor is used to create a new upload status monitor, note that uploadStatusMap and orderedFileList are used to maintain the state of the monitor and will change as the underlying map/list changes.

Types

type ApiOpts added in v1.1.3 

type ApiOpts struct {
	api.SecurityTokenInterface
	api.FileInterface
}

type DummyMonitor added in v1.1.3 

type DummyMonitor struct {
	ManualQuit bool
}

func (*DummyMonitor) Init added in v1.1.3 

func (m *DummyMonitor) Init() tea.Cmd

func (*DummyMonitor) Quit added in v1.1.3 

func (m *DummyMonitor) Quit() bool

func (*DummyMonitor) Update added in v1.1.3 

func (m *DummyMonitor) Update(msg tea.Msg) (tea.Model, tea.Cmd)

func (*DummyMonitor) View added in v1.1.3 

func (m *DummyMonitor) View() string

type FileInfo 

type FileInfo struct {
	Path     string
	Size     int64
	Sha256   string
	Uploaded int64
	Status   UploadStatusEnum
}

FileInfo contains the path, size and sha256 of a file.

type FileOpts added in v1.1.3 

type FileOpts struct {
	Path string

	Recursive     bool
	IncludeHidden bool
	// contains filtered or unexported fields
}

func (*FileOpts) Valid added in v1.1.3 

func (opt *FileOpts) Valid() error

type IntHeap added in v1.1.0 

type IntHeap []int

An IntHeap is a min-heap of ints.

func NewHeap added in v1.1.0 

func NewHeap(s []int) *IntHeap

func (IntHeap) Len added in v1.1.0 

func (h IntHeap) Len() int

func (IntHeap) Less added in v1.1.0 

func (h IntHeap) Less(i, j int) bool

func (*IntHeap) Peek added in v1.1.0 

func (h *IntHeap) Peek() int

func (*IntHeap) Pop added in v1.1.0 

func (h *IntHeap) Pop() any

func (*IntHeap) Push added in v1.1.0 

func (h *IntHeap) Push(x any)

func (*IntHeap) Remove added in v1.1.0 

func (h *IntHeap) Remove(x any)

func (IntHeap) Swap added in v1.1.0 

func (h IntHeap) Swap(i, j int)

type MultipartOpts added in v1.1.0 

type MultipartOpts struct {
	Threads uint
	Size    string
}

func (*MultipartOpts) Valid added in v1.1.0 

func (opt *MultipartOpts) Valid() error

type TickMsg added in v1.1.4 

type TickMsg time.Time

type UploadDB added in v1.1.0 

type UploadDB struct {
	*bolt.DB
}

func NewUploadDB added in v1.1.0 

func NewUploadDB(filename string, recordId string, hash string) (*UploadDB, error)

func (*UploadDB) BatchPut added in v1.1.0 

func (db *UploadDB) BatchPut(kvs map[string][]byte) error

BatchPut puts a batch of key-value pairs into the database.

func (*UploadDB) Delete added in v1.1.0 

func (db *UploadDB) Delete() error

Delete removes the database file from the filesystem.

func (*UploadDB) Get added in v1.1.0 

func (db *UploadDB) Get(key string) ([]byte, error)

Get retrieves the value of a key from the database. if the key does not exist, it returns nil.

func (*UploadDB) Reset added in v1.1.1 

func (db *UploadDB) Reset() error

Reset removes all the keys from the database multipart_uploads bucket.

type UploadManager 

type UploadManager struct {
	Errs map[string]error
	sync.WaitGroup
	// contains filtered or unexported fields
}

UploadManager is a manager for uploading files through minio client. Note that it's user's responsibility to check the Errs field after Wait() to see if there's any error.

func NewUploadManagerFromConfig added in v1.1.0 

func NewUploadManagerFromConfig(proj *name.Project, timeout time.Duration, hideMonitor bool, apiOpts *ApiOpts, multiOpts *MultipartOpts) (*UploadManager, error)

func (*UploadManager) AddErr 

func (um *UploadManager) AddErr(path string, err error)

AddErr adds an error to the manager.

func (*UploadManager) AddFile added in v1.1.4 

func (um *UploadManager) AddFile(path string)

AddFile adds a file to the upload manager.

func (*UploadManager) Debugf added in v1.1.0 

func (um *UploadManager) Debugf(format string, args ...interface{})

func (*UploadManager) FMultipartPutObject 

func (um *UploadManager) FMultipartPutObject(ctx context.Context, bucket string, key string, filePath string, fileSize int64, fileSha256 string, opts minio.PutObjectOptions) (err error)

func (*UploadManager) FPutObject 

func (um *UploadManager) FPutObject(absPath string, bucket string, key string, userTags map[string]string)

FPutObject uploads a file to a bucket with a key and sha256. If the file size is larger than minPartSize, it will use multipart upload.

func (*UploadManager) PrintErrs added in v1.1.0 

func (um *UploadManager) PrintErrs()

PrintErrs prints all errors.

func (*UploadManager) Run added in v1.1.3 

func (um *UploadManager) Run(ctx context.Context, rcd *name.Record, fileOpts *FileOpts) error

func (*UploadManager) UploadFileThroughUrl added in v1.1.0 

func (um *UploadManager) UploadFileThroughUrl(file string, uploadUrl string) error

UploadFileThroughUrl uploads a single file to the given uploadUrl. um is the upload manager to use. file is the absolute path of the file to be uploaded. uploadUrl is the pre-signed url to upload the file to.

func (*UploadManager) Wait 

func (um *UploadManager) Wait()

Wait waits for all uploads to finish. And wait for status monitor to finish.

type UploadStatusEnum 

type UploadStatusEnum int

UploadStatusEnum is used to keep track of the state of a file upload 

const (
	// Unprocessed is used to indicate that the file has not been processed yet
	Unprocessed UploadStatusEnum = iota

	// PreviouslyUploaded is used to indicate that the file has been uploaded before
	PreviouslyUploaded

	// UploadInProgress is used to indicate that the file upload is in progress
	UploadInProgress

	// UploadCompleted is used to indicate that the file upload has completed
	UploadCompleted

	// MultipartCompletionInProgress is used to indicate that the multipart upload completion is in progress
	MultipartCompletionInProgress

	// UploadFailed is used to indicate that the file upload has failed
	UploadFailed
)

type UploadStatusMonitor 

type UploadStatusMonitor struct {
	ManualQuit bool
	// contains filtered or unexported fields
}

UploadStatusMonitor is a bubbletea model that is used to monitor the progress of file uploads

func (*UploadStatusMonitor) Init 

func (m *UploadStatusMonitor) Init() tea.Cmd

func (*UploadStatusMonitor) Quit added in v1.1.3 

func (m *UploadStatusMonitor) Quit() bool

func (*UploadStatusMonitor) Update 

func (m *UploadStatusMonitor) Update(msg tea.Msg) (tea.Model, tea.Cmd)

func (*UploadStatusMonitor) View 

func (m *UploadStatusMonitor) View() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.