filestore

package module
v1.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 25, 2023 License: Apache-2.0, MIT Imports: 21 Imported by: 74

README

go-filestore

a by-reference file-backed blockstore

standard-readme compliant GoDoc

❗ This repo is no longer maintained.

👉 We highly recommend switching to the maintained version at https://github.com/ipfs/boxo/tree/main/filestore. 🏎️ Good news! There is tooling and documentation to expedite a switch in your repo.

⚠️ If you continue using this repo, please note that security fixes will not be provided (unless someone steps in to maintain it).

📚 Learn more, including how to take the maintainership mantle or ask questions, here.

Table of Contents

Documentation

https://godoc.org/github.com/ipfs/go-filestore

Want to hack on IPFS?

License

MIT

Documentation

Overview

Package filestore implements a Blockstore which is able to read certain blocks of data directly from its original location in the filesystem.

In a Filestore, object leaves are stored as FilestoreNodes. FilestoreNodes include a filesystem path and an offset, allowing a Blockstore dealing with such blocks to avoid storing the whole contents and reading them from their filesystem location instead.

Index

Constants

This section is empty.

Variables

View Source 
var ErrFilestoreNotEnabled = errors.New("filestore is not enabled, see https://git.io/vNItf")

Deprecated: use github.com/ipfs/boxo/filestore.ErrFilestoreNotEnabled

View Source 
var ErrUrlstoreNotEnabled = errors.New("urlstore is not enabled")

Deprecated: use github.com/ipfs/boxo/filestore.ErrUrlstoreNotEnabled

View Source 
var FilestorePrefix = ds.NewKey("filestore")

FilestorePrefix identifies the key prefix for FileManager blocks.

Deprecated: use github.com/ipfs/boxo/filestore.FilestorePrefix

Functions

func IsURL deprecated 

func IsURL(str string) bool

IsURL returns true if the string represents a valid URL that the urlstore can handle. More specifically it returns true if a string begins with 'http://' or 'https://'.

Deprecated: use github.com/ipfs/boxo/filestore.IsURL

func ListAll deprecated 

func ListAll(ctx context.Context, fs *Filestore, fileOrder bool) (func(context.Context) *ListRes, error)

ListAll returns a function as an iterator which, once invoked, returns one by one each block in the Filestore's FileManager. ListAll does not verify that the references are valid or whether the raw data is accessible. See VerifyAll().

Deprecated: use github.com/ipfs/boxo/filestore.ListAll

func VerifyAll deprecated 

func VerifyAll(ctx context.Context, fs *Filestore, fileOrder bool) (func(context.Context) *ListRes, error)

VerifyAll returns a function as an iterator which, once invoked, returns one by one each block in the Filestore's FileManager. VerifyAll checks that the reference is valid and that the block data can be read.

Deprecated: use github.com/ipfs/boxo/filestore.VerifyAll

Types

type CorruptReferenceError deprecated 

type CorruptReferenceError struct {
	Code Status
	Err  error
}

CorruptReferenceError implements the error interface. It is used to indicate that the block contents pointed by the referencing blocks cannot be retrieved (i.e. the file is not found, or the data changed as it was being read).

Deprecated: use github.com/ipfs/boxo/filestore.CorruptReferenceError

func (CorruptReferenceError) Error 

func (c CorruptReferenceError) Error() string

Error() returns the error message in the CorruptReferenceError as a string.

type FileManager deprecated 

type FileManager struct {
	AllowFiles bool
	AllowUrls  bool
	// contains filtered or unexported fields
}

FileManager is a blockstore implementation which stores special blocks FilestoreNode type. These nodes only contain a reference to the actual location of the block data in the filesystem (a path and an offset).

Deprecated: use github.com/ipfs/boxo/filestore.FileManager

func NewFileManager deprecated 

func NewFileManager(ds ds.Batching, root string) *FileManager

NewFileManager initializes a new file manager with the given datastore and root. All FilestoreNodes paths are relative to the root path given here, which is prepended for any operations.

Deprecated: use github.com/ipfs/boxo/filestore.NewFileManager

func (*FileManager) AllKeysChan 

func (f *FileManager) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)

AllKeysChan returns a channel from which to read the keys stored in the FileManager. If the given context is cancelled the channel will be closed.

All CIDs returned are of type Raw.

func (*FileManager) DeleteBlock 

func (f *FileManager) DeleteBlock(ctx context.Context, c cid.Cid) error

DeleteBlock deletes the reference-block from the underlying datastore. It does not touch the referenced data.

func (*FileManager) Get 

func (f *FileManager) Get(ctx context.Context, c cid.Cid) (blocks.Block, error)

Get reads a block from the datastore. Reading a block is done in two steps: the first step retrieves the reference block from the datastore. The second step uses the stored path and offsets to read the raw block data directly from disk.

func (*FileManager) GetSize 

func (f *FileManager) GetSize(ctx context.Context, c cid.Cid) (int, error)

GetSize gets the size of the block from the datastore.

This method may successfully return the size even if returning the block would fail because the associated file is no longer available.

func (*FileManager) Has 

func (f *FileManager) Has(ctx context.Context, c cid.Cid) (bool, error)

Has returns if the FileManager is storing a block reference. It does not validate the data, nor checks if the reference is valid.

func (*FileManager) Put 

func (f *FileManager) Put(ctx context.Context, b *posinfo.FilestoreNode) error

Put adds a new reference block to the FileManager. It does not check that the reference is valid.

func (*FileManager) PutMany 

func (f *FileManager) PutMany(ctx context.Context, bs []*posinfo.FilestoreNode) error

PutMany is like Put() but takes a slice of blocks instead, allowing it to create a batch transaction.

type Filestore deprecated 

type Filestore struct {
	// contains filtered or unexported fields
}

Filestore implements a Blockstore by combining a standard Blockstore to store regular blocks and a special Blockstore called FileManager to store blocks which data exists in an external file.

Deprecated: use github.com/ipfs/boxo/filestore.Filestore

func NewFilestore deprecated 

func NewFilestore(bs blockstore.Blockstore, fm *FileManager) *Filestore

NewFilestore creates one using the given Blockstore and FileManager.

Deprecated: use github.com/ipfs/boxo/filestore.NewFilestore

func (*Filestore) AllKeysChan 

func (f *Filestore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)

AllKeysChan returns a channel from which to read the keys stored in the blockstore. If the given context is cancelled the channel will be closed.

func (*Filestore) DeleteBlock 

func (f *Filestore) DeleteBlock(ctx context.Context, c cid.Cid) error

DeleteBlock deletes the block with the given key from the blockstore. As expected, in the case of FileManager blocks, only the reference is deleted, not its contents. It may return ErrNotFound when the block is not stored.

func (*Filestore) FileManager 

func (f *Filestore) FileManager() *FileManager

FileManager returns the FileManager in Filestore.

func (*Filestore) Get 

func (f *Filestore) Get(ctx context.Context, c cid.Cid) (blocks.Block, error)

Get retrieves the block with the given Cid. It may return ErrNotFound when the block is not stored.

func (*Filestore) GetSize 

func (f *Filestore) GetSize(ctx context.Context, c cid.Cid) (int, error)

GetSize returns the size of the requested block. It may return ErrNotFound when the block is not stored.

func (*Filestore) Has 

func (f *Filestore) Has(ctx context.Context, c cid.Cid) (bool, error)

Has returns true if the block with the given Cid is stored in the Filestore.

func (*Filestore) HashOnRead 

func (f *Filestore) HashOnRead(enabled bool)

HashOnRead calls blockstore.HashOnRead.

func (*Filestore) MainBlockstore 

func (f *Filestore) MainBlockstore() blockstore.Blockstore

MainBlockstore returns the standard Blockstore in the Filestore.

func (*Filestore) Put 

func (f *Filestore) Put(ctx context.Context, b blocks.Block) error

Put stores a block in the Filestore. For blocks of underlying type FilestoreNode, the operation is delegated to the FileManager, while the rest of blocks are handled by the regular blockstore.

func (*Filestore) PutMany 

func (f *Filestore) PutMany(ctx context.Context, bs []blocks.Block) error

PutMany is like Put(), but takes a slice of blocks, allowing the underlying blockstore to perform batch transactions.

type ListRes deprecated 

type ListRes struct {
	Status   Status
	ErrorMsg string
	Key      cid.Cid
	FilePath string
	Offset   uint64
	Size     uint64
}

ListRes wraps the response of the List*() functions, which allows to obtain and verify blocks stored by the FileManager of a Filestore. It includes information about the referenced block.

Deprecated: use github.com/ipfs/boxo/filestore.ListRes

func List deprecated 

func List(ctx context.Context, fs *Filestore, key cid.Cid) *ListRes

List fetches the block with the given key from the Filemanager of the given Filestore and returns a ListRes object with the information. List does not verify that the reference is valid or whether the raw data is accesible. See Verify().

Deprecated: use github.com/ipfs/boxo/filestore.List

func Verify deprecated 

func Verify(ctx context.Context, fs *Filestore, key cid.Cid) *ListRes

Verify fetches the block with the given key from the Filemanager of the given Filestore and returns a ListRes object with the information. Verify makes sure that the reference is valid and the block data can be read.

Deprecated: use github.com/ipfs/boxo/filestore.Verify

func (*ListRes) FormatLong 

func (r *ListRes) FormatLong(enc func(cid.Cid) string) string

FormatLong returns a human readable string for a ListRes object

type Status deprecated 

type Status int32

Status is used to identify the state of the block data referenced by a FilestoreNode. Among other places, it is used by CorruptReferenceError.

Deprecated: use github.com/ipfs/boxo/filestore.Status 

const (
	// Deprecated: use github.com/ipfs/boxo/filestore.StatusOk
	StatusOk Status = 0
	// Deprecated: use github.com/ipfs/boxo/filestore.StatusFileError
	StatusFileError Status = 10 // Backing File Error
	// Deprecated: use github.com/ipfs/boxo/filestore.StatusFileNotFound
	StatusFileNotFound Status = 11 // Backing File Not Found
	// Deprecated: use github.com/ipfs/boxo/filestore.StatusFileChanged
	StatusFileChanged Status = 12 // Contents of the file changed
	// Deprecated: use github.com/ipfs/boxo/filestore.StatusOtherError
	StatusOtherError Status = 20 // Internal Error, likely corrupt entry
	// Deprecated: use github.com/ipfs/boxo/filestore.StatusKeyNotFound
	StatusKeyNotFound Status = 30
)

These are the supported Status codes.

func (Status) Format 

func (s Status) Format() string

Format returns the status formatted as a string with leading 0s.

func (Status) String 

func (s Status) String() string

String provides a human-readable representation for Status codes.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.