helpers

package
v1.28.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 3, 2024 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package helpers provide useful wrappers for clients using the lakeFS OpenAPI.

Index

Constants

View Source
const (
	MinDownloadPartSize        int64 = 1024 * 64       // 64KB
	DefaultDownloadPartSize    int64 = 1024 * 1024 * 8 // 8MB
	DefaultDownloadConcurrency       = 10
)
View Source
const DefaultUploadConcurrency = 5

DefaultUploadConcurrency is the default number of goroutines to spin up when uploading a multipart upload

View Source
const DefaultUploadPartSize = MinUploadPartSize

DefaultUploadPartSize is the default part size to buffer chunks of a payload into

View Source
const MaxUploadParts int32 = 10000

MaxUploadParts is the maximum allowed number of parts in a multipart upload

View Source
const MinUploadPartSize int64 = 1024 * 1024 * 5

MinUploadPartSize is the minimum allowed part size when uploading a part

Variables

View Source
var (
	// ErrUnsupportedProtocol is the error returned when lakeFS server requests the client
	// use a protocol it does not know about.  Until recompiled to use a newer client,
	// upload the object using through the lakeFS server.
	ErrUnsupportedProtocol = errors.New("unsupported protocol")

	// ErrRequestFailed is an error returned for failing lakeFS server replies.
	ErrRequestFailed = errors.New("request failed")
	ErrConflict      = errors.New("conflict")
)

Functions

func ClientUpload

func ClientUpload(ctx context.Context, client apigen.ClientWithResponsesInterface, repoID, branchID, objPath string, metadata map[string]string, contentType string, contents io.ReadSeeker) (*apigen.ObjectStats, error)

ClientUpload uploads contents as a file via lakeFS

func ClientUploadPreSign added in v0.91.0

func ClientUploadPreSign(ctx context.Context, client apigen.ClientWithResponsesInterface, httpClient *http.Client, repoID, branchID, objPath string, metadata map[string]string, contentType string, contents io.ReadSeeker, presignMultipartSupport bool) (*apigen.ObjectStats, error)

func HTTPResponseAsError added in v0.58.0

func HTTPResponseAsError(httpResponse *http.Response) error

func ResponseAsError

func ResponseAsError(response interface{}) error

ResponseAsError returns a UserVisibleAPIError wrapping an ErrRequestFailed wrapping a response from the server. It searches for a non-nil unsuccessful HTTPResponse field and uses its message, along with a Body that it assumes is an api.Error.

Types

type APIFields added in v0.53.1

type APIFields struct {
	StatusCode int
	Status     string
	Message    string
}

APIFields are fields to use to format an HTTP error response that can be shown to the user.

type CallFailedError added in v0.53.1

type CallFailedError struct {
	Err     error
	Message string
}

CallFailedError is an error performing the HTTP request itself formatted to be shown to a user. It does _not_ update its message when wrapped so usually should not be wrapped.

func (CallFailedError) Error added in v0.53.1

func (e CallFailedError) Error() string

func (CallFailedError) Unwrap added in v0.53.1

func (e CallFailedError) Unwrap() error

type Downloader added in v1.9.0

type Downloader struct {
	Client     *apigen.ClientWithResponses
	PreSign    bool
	HTTPClient *http.Client
	PartSize   int64
}

func NewDownloader added in v1.9.0

func NewDownloader(client *apigen.ClientWithResponses, preSign bool) *Downloader

func (*Downloader) Download added in v1.9.0

func (d *Downloader) Download(ctx context.Context, src uri.URI, dst string) error

Download downloads an object from lakeFS to a local file, create the destination directory if needed.

type PreSignUploader added in v1.7.0

type PreSignUploader struct {
	Concurrency      int
	HTTPClient       *http.Client
	Client           apigen.ClientWithResponsesInterface
	MultipartSupport bool
}

PreSignUploader uploads contents as a file via lakeFS using presigned urls It supports both multipart and single part uploads.

func NewPreSignUploader added in v1.7.0

func NewPreSignUploader(client apigen.ClientWithResponsesInterface, httpClient *http.Client, multipartSupport bool) *PreSignUploader

func (*PreSignUploader) Upload added in v1.7.0

func (u *PreSignUploader) Upload(ctx context.Context, repoID string, branchID string, objPath string, content io.ReadSeeker,
	contentType string, metadata map[string]string,
) (*apigen.ObjectStats, error)

type UserVisibleAPIError added in v0.53.1

type UserVisibleAPIError struct {
	APIFields
	Err error
}

UserVisibleAPIError is an HTTP error response formatted to be shown to a user. It does _not_ update its message when wrapped so usually should not be wrapped.

func (UserVisibleAPIError) Error added in v0.53.1

func (e UserVisibleAPIError) Error() string

func (UserVisibleAPIError) Unwrap added in v0.53.1

func (e UserVisibleAPIError) Unwrap() error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL