backblaze

package module
v0.0.0-...-aab6480 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 30, 2019 License: MIT Imports: 17 Imported by: 0

README

go-backblaze

GoDoc Build Status

A golang client for Backblaze's B2 storage

Usage

Some simple examples to get you started. Errors are ommitted for brevity

Import the API package

import "gopkg.in/kothar/go-backblaze.v0"

Create an API client

b2, _ := backblaze.NewB2(backblaze.Credentials{
  AccountID:      accountID,
  ApplicationKey: applicationKey,
})

Create a bucket

bucket, _ := b2.CreateBucket("test_bucket", backblaze.AllPrivate)

Uploading a file

reader, _ := os.Open(path)
name := filepath.Base(path)
metadata := make(map[string]string)

file, _ := bucket.UploadFile(name, metadata, reader)

All API methods except B2.AuthorizeAccount and Bucket.UploadHashedFile will retry once if authorization fails, which allows the operation to proceed if the current authorization token has expired.

To disable this behaviour, set B2.NoRetry to true

b2 command line client

A test applicaiton has been implemented using this package, and can be found in the /b2 directory. It should provide you with more examples of how to use the API in your own applications.

To install the b2 command, use:

go get -u gopkg.in/kothar/go-backblaze.v0/b2

$ b2 --help
Usage:
  b2 [OPTIONS] <command>

Application Options:
      --account= The account ID to use [$B2_ACCOUNT_ID]
      --appKey=  The application key to use [$B2_APP_KEY]
  -b, --bucket=  The bucket to access [$B2_BUCKET]
  -d, --debug    Debug API requests
  -v, --verbose  Display verbose output

Help Options:
  -h, --help     Show this help message

Available commands:
  createbucket  Create a new bucket
  delete        Delete a file
  deletebucket  Delete a bucket
  get           Download a file
  list          List files in a bucket
  listbuckets   List buckets in an account
  put           Store a file

Documentation

Overview

Package backblaze B2 API for Golang

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Allowed 

type Allowed struct {
	Capabilities []string `json:"capabilities"`
	BucketID     *string  `json:"bucketId,omitempty"`
	BucketName   *string  `json:"bucketName,omitempty"`
	NamePrefix   *string  `json:"namePrefix,omitempty"`
}

Allowed contains the Capabilities of an auth token

type B2 

type B2 struct {
	Credentials

	// Capabilities of the auth token
	Allowed

	//LargeFile Information
	LargeFileInfo

	// If true, don't retry requests if authorization has expired
	NoRetry bool

	// If true, display debugging information about API calls
	Debug bool

	// Number of MaxIdleUploads to keep for reuse.
	// This must be set prior to creating a bucket struct
	MaxIdleUploads int
	// contains filtered or unexported fields
}

B2 implements a B2 API client. Do not modify state concurrently.

Example 
NewB2(Credentials{
	AccountID:      "", // Obtained from your B2 account page.
	ApplicationKey: "", // Obtained from your B2 account page.
})

Output:

func NewB2 

func NewB2(creds Credentials) (*B2, error)

NewB2 creates a new Client for accessing the B2 API. The AuthorizeAccount method will be called immediately.

func (*B2) AuthorizeAccount 

func (c *B2) AuthorizeAccount() error

AuthorizeAccount is used to log in to the B2 API.

func (*B2) Bucket 

func (b *B2) Bucket(bucketRequest *GetBucketRequest) (*Bucket, error)

Bucket looks up a bucket for the currently authorized client

func (*B2) CreateBucket 

func (b *B2) CreateBucket(bucketName string, bucketType BucketType) (*Bucket, error)

CreateBucket creates a new B2 Bucket in the authorized account.

Buckets can be named. The name must be globally unique. No account can use a bucket with the same name. Buckets are assigned a unique bucketId which is used when uploading, downloading, or deleting files.

func (*B2) CreateBucketWithInfo 

func (b *B2) CreateBucketWithInfo(bucketName string, bucketType BucketType, bucketInfo map[string]string, lifecycleRules []LifecycleRule) (*Bucket, error)

CreateBucketWithInfo extends CreateBucket to add bucket info and lifecycle rules to the creation request

func (*B2) DownloadFileByID 

func (c *B2) DownloadFileByID(fileID string) (*File, io.ReadCloser, error)

DownloadFileByID downloads a file from B2 using its unique ID

func (*B2) DownloadFileRangeByID 

func (c *B2) DownloadFileRangeByID(fileID string, fileRange *FileRange) (*File, io.ReadCloser, error)

DownloadFileRangeByID downloads part of a file from B2 using its unique ID and a requested byte range.

func (*B2) DownloadURL 

func (c *B2) DownloadURL() (string, error)

DownloadURL returns the URL prefix needed to construct download links. Bucket.FileURL will costruct a full URL for given file names.

func (*B2) FinishLargeFile 

func (c *B2) FinishLargeFile(fileID string, partSha1Array []string) (*FinishLargeFileResponse, error)

FinishLargeFile converts the parts uploaded from b2_get_upload_part_url into the final file

func (*B2) ListBuckets 

func (b *B2) ListBuckets() ([]*Bucket, error)

ListBuckets lists buckets associated with an account, in alphabetical order by bucket ID.

func (*B2) ReadaheadFile 

func (c *B2) ReadaheadFile(file *File) (io.ReadCloser, error)

ReadaheadFile attempts to load chunks of the file being downloaded ahead of time to improve transfer rates. This method attempts to optimise the chunk size based on the file size and a target of 15 workers

File ranges are downloaded using Content-Range requests. See DownloadFileRangeByName

func (*B2) ReadaheadFileOptions 

func (c *B2) ReadaheadFileOptions(file *File, chunkSize, chunkAhead, numWorkers int) (io.ReadCloser, error)

ReadaheadFileOptions attempts to load chunks of the file being downloaded ahead of time to improve transfer rates. This method extends ReadaheadFile with options to configure the chunk size, number of chunks to read ahead and the number of workers to use.

File ranges are downloaded using Content-Range requests. See DownloadFileRangeByName

type B2Error 

type B2Error struct {
	Code    string `json:"code"`
	Message string `json:"message"`
	Status  int    `json:"status"`
}

B2Error encapsulates an error message returned by the B2 API.

Failures to connect to the B2 servers, and networking problems in general can cause errors

func (B2Error) Error 

func (e B2Error) Error() string

func (*B2Error) IsFatal 

func (e *B2Error) IsFatal() bool

IsFatal returns true if this error represents an error which can't be recovered from by retrying

type Bucket 

type Bucket struct {
	*BucketInfo
	// contains filtered or unexported fields
}

Bucket provides access to the files stored in a B2 Bucket

Example 
var b2 B2
// ...

b2.CreateBucket("test_bucket", AllPrivate)

Output:

func (*Bucket) Delete 

func (b *Bucket) Delete() error

Delete removes removes the bucket from the authorized account. Only buckets that contain no version of any files can be deleted.

func (*Bucket) DeleteFileVersion 

func (b *Bucket) DeleteFileVersion(fileName, fileID string) (*FileStatus, error)

DeleteFileVersion deletes one version of a file from B2.

If the version you delete is the latest version, and there are older versions, then the most recent older version will become the current version, and be the one that you'll get when downloading by name. See the File Versions page for more details.

func (*Bucket) DownloadFileByName 

func (b *Bucket) DownloadFileByName(fileName string) (*File, io.ReadCloser, error)

DownloadFileByName downloads one file by providing the name of the bucket and the name of the file.

func (*Bucket) DownloadFileRangeByName 

func (b *Bucket) DownloadFileRangeByName(fileName string, fileRange *FileRange) (*File, io.ReadCloser, error)

DownloadFileRangeByName downloads part of a file by providing the name of the bucket, the name of the file, and a requested byte range

func (*Bucket) FileURL 

func (b *Bucket) FileURL(fileName string) (string, error)

FileURL returns a URL which may be used to download the latest version of a file. This returned URL will only work for public buckets unless the correct authorization header is provided.

func (*Bucket) GetFileInfo 

func (b *Bucket) GetFileInfo(fileID string) (*File, error)

GetFileInfo retrieves information about one file stored in B2.

func (*Bucket) GetUploadAuth 

func (b *Bucket) GetUploadAuth() (*UploadAuth, error)

GetUploadAuth retrieves the URL to use for uploading files.

When you upload a file to B2, you must call b2_get_upload_url first to get the URL for uploading directly to the place where the file will be stored.

If the upload is successful, ReturnUploadAuth(*uploadAuth) should be called to place it back in the pool for reuse.

func (*Bucket) HideFile 

func (b *Bucket) HideFile(fileName string) (*FileStatus, error)

HideFile hides a file so that downloading by name will not find the file, but previous versions of the file are still stored. See File Versions about what it means to hide a file.

func (*Bucket) ListFileNames 

func (b *Bucket) ListFileNames(startFileName string, maxFileCount int) (*ListFilesResponse, error)

ListFileNames lists the names of all files in a bucket, starting at a given name.

See ListFileNamesWithPrefix

func (*Bucket) ListFileNamesWithPrefix 

func (b *Bucket) ListFileNamesWithPrefix(startFileName string, maxFileCount int, prefix, delimiter string) (*ListFilesResponse, error)

ListFileNamesWithPrefix lists the names of all files in a bucket, starting at a given name.

This call returns at most 1000 file names per transaction, but it can be called repeatedly to scan through all of the file names in a bucket. Each time you call, it returns a "nextFileName" that can be used as the starting point for the next call.

If you set maxFileCount to more than 1000 (max 10,000) and more than 1000 are returned, the call will be billed as multiple transactions, as if you had made requests in a loop asking for 1000 at a time.

Files returned will be limited to those with the given prefix. The empty string matches all files.

If a delimiter is provided, files returned will be limited to those within the top folder, or any one subfolder. Folder names will also be returned. The delimiter character will be used to "break" file names into folders.

func (*Bucket) ListFileVersions 

func (b *Bucket) ListFileVersions(startFileName, startFileID string, maxFileCount int) (*ListFileVersionsResponse, error)

ListFileVersions lists all of the versions of all of the files contained in one bucket, in alphabetical order by file name, and by reverse of date/time uploaded for versions of files with the same name.

func (*Bucket) ReadaheadFileByName 

func (b *Bucket) ReadaheadFileByName(fileName string) (*File, io.ReadCloser, error)

ReadaheadFileByName attempts to load chunks of the file being downloaded ahead of time to improve transfer rates. File ranges are downloaded using Content-Range requests. See DownloadFileRangeByName

func (*Bucket) ReturnUploadAuth 

func (b *Bucket) ReturnUploadAuth(uploadAuth *UploadAuth)

ReturnUploadAuth returns an upload URL to the available pool. This should not be called if the upload fails. Instead request another GetUploadAuth() and retry.

func (*Bucket) StartLargeFile 

func (b *Bucket) StartLargeFile(fileName string, contentType *string, fileInfo *map[string]string) (*StartLargeFileResponse, error)

StartLargeFile prepares a large_file for upload returning the file ID

func (*Bucket) Update 

func (b *Bucket) Update(bucketType BucketType) error

Update allows the bucket type to be changed

func (*Bucket) UpdateAll 

func (b *Bucket) UpdateAll(bucketType BucketType, bucketInfo map[string]string, lifecycleRules []LifecycleRule, ifRevisionIs int) error

UpdateAll allows all bucket properties to be changed

bucketType (optional) -- One of: "allPublic", "allPrivate". "allPublic" means that anybody can download the files is the bucket; "allPrivate" means that you need an authorization token to download them. If not specified, setting will remain unchanged.

bucketInfo (optional) -- User-defined information to be stored with the bucket. If specified, the existing bucket info will be replaced with the new info. If not specified, setting will remain unchanged. Cache-Control policies can be set here on a global level for all the files in the bucket.

lifecycleRules (optional) -- The list of lifecycle rules for this bucket. If specified, the existing lifecycle rules will be replaced with this new list. If not specified, setting will remain unchanged.

ifRevisionIs (optional) -- When set (> 0), the update will only happen if the revision number stored in the B2 service matches the one passed in. This can be used to avoid having simultaneous updates make conflicting changes.

func (*Bucket) UploadFile 

func (b *Bucket) UploadFile(name string, meta map[string]string, file io.Reader) (*File, error)

UploadFile calls UploadTypedFile with the b2/x-auto contentType

Example 
var bucket Bucket
// ...

path := "/path/to/file"
reader, _ := os.Open(path)
name := filepath.Base(path)
metadata := make(map[string]string)

bucket.UploadFile(name, metadata, reader)

Output:

func (*Bucket) UploadHashedFile 

func (b *Bucket) UploadHashedFile(
	name string, meta map[string]string, file io.Reader,
	sha1Hash string, contentLength int64) (*File, error)

UploadHashedFile calls UploadHashedTypedFile with the b2/x-auto file type

func (*Bucket) UploadHashedTypedFile 

func (b *Bucket) UploadHashedTypedFile(
	name, contentType string, meta map[string]string, file io.Reader,
	sha1Hash string, contentLength int64) (*File, error)

UploadHashedTypedFile Uploads a file to B2, returning its unique file ID.

This method will not retry if the upload fails, as the reader may have consumed some bytes. If the error type is B2Error and IsFatal returns false, you may retry the upload and expect it to succeed eventually.

func (*Bucket) UploadLargeFile 

func (b *Bucket) UploadLargeFile(fileName string, contentType *string, fileInfo *map[string]string, filePath string, uploadThreads int) (*FinishLargeFileResponse, error)

UploadLargeFile handles the full process of a multipart upload

func (*Bucket) UploadTypedFile 

func (b *Bucket) UploadTypedFile(name, contentType string, meta map[string]string, file io.Reader) (*File, error)

UploadTypedFile uploads a file to B2, returning its unique file ID. This method computes the hash of the file before passing it to UploadHashedFile

type BucketInfo 

type BucketInfo struct {
	// The account that the bucket is in.
	AccountID string `json:"accountId"`

	// The unique ID of the bucket.
	ID string `json:"bucketId"`

	// User-defined information to be stored with the bucket.
	Info map[string]string `json:"bucketInfo"`

	// The name to give the new bucket.
	// Bucket names must be a minimum of 6 and a maximum of 50 characters long, and must be globally unique;
	// two different B2 accounts cannot have buckets with the name name. Bucket names can consist of: letters,
	// digits, and "-". Bucket names cannot start with "b2-"; these are reserved for internal Backblaze use.
	Name string `json:"bucketName"`

	// Either "allPublic", meaning that files in this bucket can be downloaded by anybody, or "allPrivate",
	// meaning that you need a bucket authorization token to download the files.
	BucketType BucketType `json:"bucketType"`

	// The initial list of lifecycle rules for this bucket.
	LifecycleRules []LifecycleRule `json:"lifecycleRules"`

	// A counter that is updated every time the bucket is modified.
	Revision int `json:"revision"`
}

BucketInfo describes a bucket

type BucketType 

type BucketType string

BucketType defines the security setting for a bucket 

const (
	AllPublic  BucketType = "allPublic"
	AllPrivate BucketType = "allPrivate"
	Snapshot   BucketType = "snapshot"
)

Buckets can be either public, private, or snapshot

type Credentials 

type Credentials struct {
	AccountID      string
	ApplicationKey string
	KeyID          string
}

Credentials are the identification required by the Backblaze B2 API

The account ID is a 12-digit hex number that you can get from your account page on backblaze.com.

The application key is a 40-digit hex number that you can get from your account page on backblaze.com.

The key id is the application key id that you get when generating an application key. If using the master application key, leave this set to an empty string as your account id will be used instead.

type File 

type File struct {
	ID              string            `json:"fileId"`
	Name            string            `json:"fileName"`
	AccountID       string            `json:"accountId"`
	BucketID        string            `json:"bucketId"`
	ContentLength   int64             `json:"contentLength"`
	ContentSha1     string            `json:"contentSha1"`
	ContentType     string            `json:"contentType"`
	FileInfo        map[string]string `json:"fileInfo"`
	Action          FileAction        `json:"action"`
	UploadTimestamp int64             `json:"uploadTimestamp"`
}

File descibes a file stored in a B2 bucket

type FileAction 

type FileAction string

FileAction indicates the current status of a file in a B2 bucket 

const (
	Upload FileAction = "upload"
	Hide   FileAction = "hide"
)

Files can be either uploads (visible) or hidden.

Hiding a file makes it look like the file has been deleted, without removing any of the history. It adds a new version of the file that is a marker saying the file is no longer there.

type FileRange 

type FileRange struct {
	Start int64
	End   int64
}

FileRange describes a range of bytes in a file by its 0-based start and end position (inclusive)

type FileStatus 

type FileStatus struct {
	File
}

FileStatus is now identical to File in repsonses from ListFileNames and ListFileVersions

type FinishLargeFileResponse 

type FinishLargeFileResponse struct {
	AccountID       string            `json:"accountId"`
	Action          FileAction        `json:"action"`
	BucketID        string            `json:"bucketId"`
	ContentLength   int64             `json:"contentLength"`
	ContentSha1     string            `json:"contentSha1"`
	ContentType     string            `json:"contentType"`
	ID              string            `json:"fileId"`
	FileInfo        map[string]string `json:"fileInfo"`
	Name            string            `json:"fileName"`
	UploadTimestamp int64             `json:"uploadTimestamp"`
}

FinishLargeFileResponse describes the response from finish_large_file

type GetBucketRequest 

type GetBucketRequest struct {
	ID   *string `json:"bucketId,omitempty"`
	Name *string `json:"bucketName,omitempty"`
}

GetBucketRequest describes the request parameters provided to Bucket to get information on a bucket

type LargeFile 

type LargeFile struct {
	*StartLargeFileResponse
	// contains filtered or unexported fields
}

LargeFile provides access to an uploadAuthPool for the LargeFile

func (*LargeFile) GetLargeFileUploadAuth 

func (l *LargeFile) GetLargeFileUploadAuth() (*UploadAuth, error)

GetLargeFileUploadAuth retrieves the URL to use for uploading parts of a large file

When you upload a part of a large file to B2, you must call b2_get_upload_part_url first to get the URL for uploading directly to the place where the part will be stored.

If the upload is successful, ReturnLargeFileUploadAuth(*uploadAuth) should be called to place it back in the pool for reuse.

func (*LargeFile) ReturnLargeFileUploadAuth 

func (l *LargeFile) ReturnLargeFileUploadAuth(uploadAuth *UploadAuth)

ReturnUploadAuth returns an upload URL to the available pool. This should not be called if the upload fails. Instead request another GetUploadAuth() and retry.

func (*LargeFile) UploadHashedPart 

func (l *LargeFile) UploadHashedPart(sha1Hash string, contentLength int64, part *Part) (*UploadPartResponse, error)

UploadHashedPart Uploads a part of a large_file to B2, returning its Sha1 value

func (*LargeFile) UploadPart 

func (l *LargeFile) UploadPart(part *Part) (*UploadPartResponse, error)

UploadPart reterives a part from the pool and uploads it

type LargeFileInfo 

type LargeFileInfo struct {
	RecommendedPartSize     int64
	AbsoluteMinimumPartSize int64
}

LargeFileInfo contains the information required for a large file upload

type LifecycleRule 

type LifecycleRule struct {
	DaysFromUploadingToHiding int    `json:"daysFromUploadingToHiding"`
	DaysFromHidingToDeleting  int    `json:"daysFromHidingToDeleting"`
	FileNamePrefix            string `json:"fileNamePrefix"`
}

LifecycleRule instructs the B2 service to automatically hide and/or delete old files. You can set up rules to do things like delete old versions of files 30 days after a newer version was uploaded.

type ListBucketsRequest 

type ListBucketsRequest struct {
	AccountID string  `json:"accountId"`
	ID        *string `json:"bucketId,omitempty"`
	Name      *string `json:"bucketName,omitempty"`
}

ListBucketsRequest describes the request parameters provided to ListBuckets to get information on a bucket

type ListFileVersionsResponse 

type ListFileVersionsResponse struct {
	Files        []FileStatus `json:"files"`
	NextFileName string       `json:"nextFileName"`
	NextFileID   string       `json:"nextFileId"`
}

ListFileVersionsResponse lists a page of file versions stored in a B2 bucket

type ListFilesResponse 

type ListFilesResponse struct {
	Files        []FileStatus `json:"files"`
	NextFileName string       `json:"nextFileName"`
}

ListFilesResponse lists a page of files stored in a B2 bucket

type Part 

type Part struct {
	// contains filtered or unexported fields
}

Part describes a part of a LargeFile that needs to be uploaded

type StartLargeFileResponse 

type StartLargeFileResponse struct {
	AccountID       string            `json:"accountId"`
	Action          FileAction        `json:"action"`
	BucketID        string            `json:"bucketId"`
	ContentLength   int64             `json:"contentLength"`
	ContentSha1     string            `json:"contentSha1"`
	ContentType     string            `json:"contentType"`
	ID              string            `json:"fileId"`
	FileInfo        map[string]string `json:"fileInfo"`
	Name            string            `json:"fileName"`
	UploadTimestamp int64             `json:"uploadTimestamp"`
}

StartLargeFileResponse describes the response from start_large_file

type UploadAuth 

type UploadAuth struct {
	AuthorizationToken string
	UploadURL          *url.URL
	Valid              bool
}

UploadAuth encapsulates the details needed to upload a file to B2 These are pooled and must be returned when complete.

type UploadPartResponse 

type UploadPartResponse struct {
	ID              string `json:"fileId"`
	PartNumber      int    `json:"partNumber"`
	ContentLength   int64  `json:"contentLength"`
	ContentSha1     string `json:"contentSha1"`
	UploadTimestamp int64  `json:"uploadTimestamp"`
}

UploadPartResponse is the response from UploadPart

Source Files

View all Source files

Directories

Path Synopsis
A minimal client for accessing B2
 A minimal client for accessing B2

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.