blobstore

package
v0.0.0-...-3518944 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 20, 2017 License: MPL-2.0 Imports: 18 Imported by: 0

Documentation

Overview

Package blobstore implements a blob storage system with pluggable backends.

In most cases, users will want to use the gnd.la/app.App.Blobstore and gnd.la/app.Context.Blobstore helper methods to obtain a connection to the default blobstore. The default blobstore can be set using gnd.la/config.

There might be additional considerations for the backend you want to use. Please, see this package's subpackages for the available backends and the documentation about them.

File metadata must be a struct and is serialized using BSON. For more information about the BSON format and struct tags that you might use to control the serialization, see gnd.la/internal/bson.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrInvalidMetadataHash indicates that the metadata hash
	// does not match the expected value and the file is likely
	// to be corrupted.
	ErrInvalidMetadataHash = errors.New("the metadata hash is invalid")
	// ErrInvalidDataHash indicates that the data hash
	// does not match the expected value and the file is likely
	// to be corrupted.
	ErrInvalidDataHash = errors.New("the data hash is invalid")
)
View Source
var (

	// ErrNotIterable indicates that the current blobstore driver
	// does not support iteration.
	ErrNotIterable = errors.New("the blobstore driver does not support iteration")
)

Functions

This section is empty.

Types

type Blobstore

type Blobstore struct {
	// contains filtered or unexported fields
}

Blobstore represents a connection to a blobstore. Use New() to initialize a Blobsore and Blobstore.Close to close it.

func New

func New(url *config.URL) (*Blobstore, error)

New returns a new *Blobstore using the given url as its configure the URL scheme represents the driver used and the rest of the values in the URL are driver dependent. Please, see the package documentation for the available drivers and each driver sub-package for driver-specific documentation.

func (*Blobstore) Close

func (s *Blobstore) Close() error

Close closes the connection to the Blobstore.

func (*Blobstore) Create

func (s *Blobstore) Create() (*WFile, error)

Create returns a new file for writing and sets its metadata to meta (which might be nil). Note that the file should be closed by calling WFile.Close.

func (*Blobstore) CreateId

func (s *Blobstore) CreateId(id string) (*WFile, error)

CreateId works like Create, but uses the given id rather than generating a new one. If a file with the same id already exists, it's overwritten.

func (*Blobstore) Driver

func (s *Blobstore) Driver() driver.Driver

Driver returns the underlying driver

func (*Blobstore) Iter

func (s *Blobstore) Iter() (Iter, error)

Iter returns an iterator which visits all the files available in the blobstore. If the underlying driver does not support iteration, (nil, ErrNotIterable) will be returned.

func (*Blobstore) Open

func (s *Blobstore) Open(id string) (*RFile, error)

Open opens the file with the given id for reading. Note that the file should be closed by calling RFile.Close after you're done with it.

func (*Blobstore) ReadAll

func (s *Blobstore) ReadAll(id string) (data []byte, err error)

ReadAll is a shorthand for Open(f).ReadAll()

func (*Blobstore) Remove

func (s *Blobstore) Remove(id string) error

Remove deletes the file with the given id.

func (*Blobstore) Serve

func (s *Blobstore) Serve(w http.ResponseWriter, id string, rng *Range) error

Serve servers the given file by writing it to the given http.ResponseWriter. Some drivers might be able to serve the file directly from their backend. Otherwise, the file will be read from the blobstore and written to w. The rng parameter might be used for sending a partial response to the client.

func (*Blobstore) Store

func (s *Blobstore) Store(b []byte, meta interface{}) (string, error)

Store works like StoreId, but generates a new id for the file.

func (*Blobstore) StoreId

func (s *Blobstore) StoreId(id string, b []byte, meta interface{}) (string, error)

StoreId is a shorthand for storing the given data in b and the metadata in meta with the given file id. If a file with the same id exists, it's overwritten.

type Iter

type Iter interface {
	// Next returns true iff there iterator can fill
	// the provided string pointer with the next
	// file id. When the files have been returned,
	// without any errors, Next() should return false
	// and Err() should return nil. If the iteration
	// stopped due to an error, Err() should return
	// non-nil.
	Next(id *string) bool
	// Err() returns any error produced while
	// iterating the files.
	Err() error
	// Close closes the iterator. It must be called in
	// order to free its associated resources.
	Close() error
}

Iter iterates over all the files available in the blobstore. Note that iteration order is undefined.

type RFile

type RFile struct {
	// contains filtered or unexported fields
}

RFile represents a blobstore file opened for reading.

func (*RFile) Check

func (r *RFile) Check() error

Check checks the integrity of both the data and the metadata in the file. If this function returns a non-nil error, the file should be considered corrupted.

func (*RFile) Close

func (r *RFile) Close() error

Close closes the file. Once the file is closed, it might not be used again.

func (*RFile) GetMeta

func (r *RFile) GetMeta(meta interface{}) error

GetMeta retrieves the file metadata, previously stored when writing the file, into the meta argument, which must be a pointer.

func (*RFile) Id

func (r *RFile) Id() string

Id returns the unique file identifier as a string.

func (*RFile) Read

func (r *RFile) Read(p []byte) (int, error)

Read reads from the file into the p buffer. This method implements the io.Reader interface.

func (*RFile) ReadAll

func (r *RFile) ReadAll() ([]byte, error)

ReadAll is a shorthand for ioutil.ReadAll(r)

func (*RFile) Seek

func (r *RFile) Seek(offset int64, whence int) (int64, error)

Seek implements the same semantics than os.File.Seek.

func (*RFile) Size

func (r *RFile) Size() (uint64, error)

Size returns the size of the file stored file.

type Range

type Range struct {
	Start *int64
	End   *int64
}

Range represents given byte range in a file. To obtain a Range from an incoming request, use ParseRange().

func ParseRange

func ParseRange(r *http.Request) *Range

ParseRange returns a *Range from the given *http.Request if it has a well formed Range header, otherwise returns nil.

func (*Range) IsValid

func (r *Range) IsValid() bool

IsValid returns true iff r is not nil and either Start or End are non-zero.

func (*Range) Range

func (r *Range) Range() (*int64, *int64)

Range() returns the Range's Start and End, which might be nil.

func (*Range) Set

func (r *Range) Set(w http.ResponseWriter, total uint64)

Set sets the corresponding headers for this Range on the given http.ResponseWriter.

func (*Range) Size

func (r *Range) Size(total uint64) uint64

Size returns the number of bytes enclosed by this range. If the range is empty, it returns 0.

func (*Range) StatusCode

func (r *Range) StatusCode() int

StatusCode returns the HTTP response status code which should be using when serving a response with this range.

func (*Range) String

func (r *Range) String() string

type WFile

type WFile struct {
	// contains filtered or unexported fields
}

WFile represents a file in the blobstore opened for writing.

func (*WFile) Close

func (w *WFile) Close() error

Close closes the file. Once the file is closed, it might not be used again.

func (*WFile) Id

func (w *WFile) Id() string

Id returns the unique file identifier as a string.

func (*WFile) SetMeta

func (w *WFile) SetMeta(meta interface{}) error

func (*WFile) Write

func (w *WFile) Write(p []byte) (int, error)

Write writes the bytes from p into the file. This method implements the io.Writer interface.

Directories

Path Synopsis
Package chunk includes several data chunking algorithms.
Package chunk includes several data chunking algorithms.
fixed
Package fixed implements a chunker which returns chunks of fixed size (except for the last one).
Package fixed implements a chunker which returns chunks of fixed size (except for the last one).
Package driver includes the interfaces required to implement a blobstore driver.
Package driver includes the interfaces required to implement a blobstore driver.
file
Package file implements the file driver for the blobstore.
Package file implements the file driver for the blobstore.
gcs
Package gcs provides a Google Cloud Storage driver for the Blobstore.
Package gcs provides a Google Cloud Storage driver for the Blobstore.
gridfs
Package gridfs implements a GridFS driver for the blobstore.
Package gridfs implements a GridFS driver for the blobstore.
leveldb
Package leveldb implements the levelb driver for the blobstore.
Package leveldb implements the levelb driver for the blobstore.
s3
Package s3 implements an s3 driver for the blobstore.
Package s3 implements an s3 driver for the blobstore.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL