storage

Documentation

Index

• type CachingTempStore
  • func NewCachingTempStore(outWriter linking.BlockWriteOpener, tempDir string) *CachingTempStore
  • func (ttrw *CachingTempStore) Close() error
  • func (ttrw *CachingTempStore) Get(ctx context.Context, key string) ([]byte, error)
  • func (ttrw *CachingTempStore) GetStream(ctx context.Context, key string) (io.ReadCloser, error)
  • func (ttrw *CachingTempStore) Has(ctx context.Context, key string) (bool, error)
  • func (ttrw *CachingTempStore) PreloadStore() types.ReadableWritableStorage
  • func (ttrw *CachingTempStore) Put(ctx context.Context, key string, data []byte) error
• type DeferredCarWriter
  • func NewDeferredCarWriterForPath(root cid.Cid, outPath string) *DeferredCarWriter
  • func NewDeferredCarWriterForStream(root cid.Cid, outStream io.Writer) *DeferredCarWriter
  • func (dcw *DeferredCarWriter) BlockWriteOpener() linking.BlockWriteOpener
  • func (dcw *DeferredCarWriter) Close() error
  • func (dcw *DeferredCarWriter) Has(ctx context.Context, key string) (bool, error)
  • func (dcw *DeferredCarWriter) OnPut(cb func(int), once bool)
  • func (dcw *DeferredCarWriter) Put(ctx context.Context, key string, content []byte) error
• type DeferredStorageCar
  • func NewDeferredStorageCar(tempDir string) *DeferredStorageCar
  • func (dcs *DeferredStorageCar) Close() error
  • func (dcs *DeferredStorageCar) Get(ctx context.Context, key string) ([]byte, error)
  • func (dcs *DeferredStorageCar) GetStream(ctx context.Context, key string) (io.ReadCloser, error)
  • func (dcs *DeferredStorageCar) Has(ctx context.Context, key string) (bool, error)
  • func (dcs *DeferredStorageCar) Put(ctx context.Context, key string, data []byte) error
• type ReadableWritableStorage

Types

type CachingTempStore

type CachingTempStore struct {
	// contains filtered or unexported fields
}

CachingTempStore is a ReadableWritableStorage that is intended for temporary use. It uses DeferredStorageCar as a backing store, so the underlying CAR file is lazily created on the first write (none will be created if there are no writes).

A provided BlockWriteOpener will receive blocks for each Put operation, this is intended to be used to write a properly ordered CARv1 file.

PreloadStore returns a secondary ReadableWritableStorage that can be used by a traversal preloader to optimistically load blocks into the temporary store. Blocks loaded via the PreloadStore will not be written to the provided BlockWriteOpener until they appear in a Put operation on the parent store. In this way, the BlockWriteOpener will receive blocks in the order that they appear in the traversal.

func NewCachingTempStore

func NewCachingTempStore(outWriter linking.BlockWriteOpener, tempDir string) *CachingTempStore

func (*CachingTempStore) Close

func (ttrw *CachingTempStore) Close() error

Close will clean up any temporary resources used by the storage.

func (*CachingTempStore) Get

func (ttrw *CachingTempStore) Get(ctx context.Context, key string) ([]byte, error)

func (*CachingTempStore) GetStream

func (ttrw *CachingTempStore) GetStream(ctx context.Context, key string) (io.ReadCloser, error)

func (*CachingTempStore) Has

func (ttrw *CachingTempStore) Has(ctx context.Context, key string) (bool, error)

func (*CachingTempStore) PreloadStore

func (ttrw *CachingTempStore) PreloadStore() types.ReadableWritableStorage

func (*CachingTempStore) Put

func (ttrw *CachingTempStore) Put(ctx context.Context, key string, data []byte) error

Put writes both to temporary readwrite caching storage (available for read operations) and to the underlying write-only CARv1 output at the same time.

type DeferredCarWriter

type DeferredCarWriter struct {
	// contains filtered or unexported fields
}

DeferredCarWriter creates a write-only CARv1 either to an existing stream or to a file designated by a supplied path. CARv1 content (including header) only begins when the first Put() operation is performed. If the output is a file, it will be created when the first Put() operation is performed. DeferredCarWriter is threadsafe, and can be used concurrently. Closing the writer will close, but not delete, the underlying file. This writer is intended for constructing the final output CARv1 for the user.

func NewDeferredCarWriterForPath

func NewDeferredCarWriterForPath(root cid.Cid, outPath string) *DeferredCarWriter

NewDeferredCarWriterForPath creates a DeferredCarWriter that will write to a file designated by the supplied path. The file will only be created on the first Put() operation.

func NewDeferredCarWriterForStream

func NewDeferredCarWriterForStream(root cid.Cid, outStream io.Writer) *DeferredCarWriter

NewDeferredCarWriterForStream creates a DeferredCarWriter that will write to the supplied stream. The stream will only be written to on the first Put() operation.

func (*DeferredCarWriter) BlockWriteOpener

func (dcw *DeferredCarWriter) BlockWriteOpener() linking.BlockWriteOpener

BlockWriteOpener returns a BlockWriteOpener that operates on this storage.

func (*DeferredCarWriter) Close

func (dcw *DeferredCarWriter) Close() error

Close closes the underlying file, if one was created.

func (*DeferredCarWriter) Has

func (dcw *DeferredCarWriter) Has(ctx context.Context, key string) (bool, error)

Has returns false if the key was not already written to the CARv1 output.

func (*DeferredCarWriter) OnPut

func (dcw *DeferredCarWriter) OnPut(cb func(int), once bool)

OnPut will call a callback when each Put() operation is started. The argument to the callback is the number of bytes being written. If once is true, the callback will be removed after the first call.

func (*DeferredCarWriter) Put

func (dcw *DeferredCarWriter) Put(ctx context.Context, key string, content []byte) error

Put writes the given content to the CARv1 output stream, creating it if it doesn't exist yet.

type DeferredStorageCar

type DeferredStorageCar struct {
	// contains filtered or unexported fields
}

DeferredStorageCar is a wrapper around github.com/ipld/go-car/v2/storage.StorageCar that defers creating the CAR until the first Put() operation. In this way it can be optimistically instantiated and no file will be created if it is never written to (such as in the case of an error).

func NewDeferredStorageCar

func NewDeferredStorageCar(tempDir string) *DeferredStorageCar

NewDeferredStorageCar creates a new DeferredStorageCar.

func (*DeferredStorageCar) Close

func (dcs *DeferredStorageCar) Close() error

Close will clean up any temporary resources used by the storage.

func (*DeferredStorageCar) Get

func (dcs *DeferredStorageCar) Get(ctx context.Context, key string) ([]byte, error)

Get returns data from the underlying CARv1.

func (*DeferredStorageCar) GetStream

func (dcs *DeferredStorageCar) GetStream(ctx context.Context, key string) (io.ReadCloser, error)

GetStream returns data from the underlying CARv1.

func (*DeferredStorageCar) Has

func (dcs *DeferredStorageCar) Has(ctx context.Context, key string) (bool, error)

Has returns true if the underlying CARv1 has the key.

func (*DeferredStorageCar) Put

func (dcs *DeferredStorageCar) Put(ctx context.Context, key string, data []byte) error

Put writes data to the underlying CARv1 which will be initialised on the first call to Put.

type ReadableWritableStorage

type ReadableWritableStorage interface {
	ipldstorage.ReadableStorage
	ipldstorage.WritableStorage
	ipldstorage.StreamingReadableStorage
}