testblobs

package
v1.109.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 22, 2024 License: AGPL-3.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewLimitedSpaceDB 

func NewLimitedSpaceDB(log *zap.Logger, db storagenode.DB, freeSpace int64) storagenode.DB

NewLimitedSpaceDB creates a new storage node DB with limited free space.

Types

type BadBlobs 

type BadBlobs struct {
	// contains filtered or unexported fields
}

BadBlobs implements a bad blob store.

func (*BadBlobs) CheckWritability 

func (bad *BadBlobs) CheckWritability(ctx context.Context) error

CheckWritability tests writability of the storage directory by creating and deleting a file.

func (*BadBlobs) Close 

func (bad *BadBlobs) Close() error

Close closes the blob store and any resources associated with it.

func (*BadBlobs) Create 

func (bad *BadBlobs) Create(ctx context.Context, ref blobstore.BlobRef) (blobstore.BlobWriter, error)

Create creates a new blob that can be written optionally takes a size argument for performance improvements, -1 is unknown size.

func (*BadBlobs) CreateVerificationFile 

func (bad *BadBlobs) CreateVerificationFile(ctx context.Context, id storj.NodeID) error

CreateVerificationFile creates a file to be used for storage directory verification.

func (*BadBlobs) Delete 

func (bad *BadBlobs) Delete(ctx context.Context, ref blobstore.BlobRef) error

Delete deletes the blob with the namespace and key.

func (*BadBlobs) DeleteNamespace 

func (bad *BadBlobs) DeleteNamespace(ctx context.Context, ref []byte) (err error)

DeleteNamespace deletes blobs of specific satellite, used after successful GE only.

func (*BadBlobs) DeleteTrashNamespace added in v1.88.2 

func (bad *BadBlobs) DeleteTrashNamespace(ctx context.Context, namespace []byte) error

DeleteTrashNamespace deletes the trash folder for the namespace.

func (*BadBlobs) DeleteWithStorageFormat 

func (bad *BadBlobs) DeleteWithStorageFormat(ctx context.Context, ref blobstore.BlobRef, formatVer blobstore.FormatVersion) error

DeleteWithStorageFormat deletes the blob with the namespace, key, and format version.

func (*BadBlobs) DiskInfo added in v1.98.1 

func (bad *BadBlobs) DiskInfo(ctx context.Context) (blobstore.DiskInfo, error)

DiskInfo returns information about the disk.

func (*BadBlobs) EmptyTrash 

func (bad *BadBlobs) EmptyTrash(ctx context.Context, namespace []byte, trashedBefore time.Time) (int64, [][]byte, error)

EmptyTrash empties the trash.

func (*BadBlobs) ListNamespaces 

func (bad *BadBlobs) ListNamespaces(ctx context.Context) ([][]byte, error)

ListNamespaces returns all namespaces that might be storing data.

func (*BadBlobs) Open 

func (bad *BadBlobs) Open(ctx context.Context, ref blobstore.BlobRef) (blobstore.BlobReader, error)

Open opens a reader with the specified namespace and key.

func (*BadBlobs) OpenWithStorageFormat 

func (bad *BadBlobs) OpenWithStorageFormat(ctx context.Context, ref blobstore.BlobRef, formatVer blobstore.FormatVersion) (blobstore.BlobReader, error)

OpenWithStorageFormat opens a reader for the already-located blob, avoiding the potential need to check multiple storage formats to find the blob.

func (*BadBlobs) RestoreTrash 

func (bad *BadBlobs) RestoreTrash(ctx context.Context, namespace []byte) ([][]byte, error)

RestoreTrash restores all files in the trash.

func (*BadBlobs) SetCheckError 

func (bad *BadBlobs) SetCheckError(err error)

SetCheckError configures the blob store to return a specific error for verification operations.

func (*BadBlobs) SetError 

func (bad *BadBlobs) SetError(err error)

SetError configures the blob store to return a specific error for all operations, except verification.

func (*BadBlobs) SpaceUsedForBlobs 

func (bad *BadBlobs) SpaceUsedForBlobs(ctx context.Context) (int64, error)

SpaceUsedForBlobs adds up how much is used in all namespaces.

func (*BadBlobs) SpaceUsedForBlobsInNamespace 

func (bad *BadBlobs) SpaceUsedForBlobsInNamespace(ctx context.Context, namespace []byte) (int64, error)

SpaceUsedForBlobsInNamespace adds up how much is used in the given namespace.

func (*BadBlobs) SpaceUsedForTrash 

func (bad *BadBlobs) SpaceUsedForTrash(ctx context.Context) (int64, error)

SpaceUsedForTrash adds up how much is used in all namespaces.

func (*BadBlobs) Stat 

func (bad *BadBlobs) Stat(ctx context.Context, ref blobstore.BlobRef) (blobstore.BlobInfo, error)

Stat looks up disk metadata on the blob file.

func (*BadBlobs) StatWithStorageFormat 

func (bad *BadBlobs) StatWithStorageFormat(ctx context.Context, ref blobstore.BlobRef, formatVer blobstore.FormatVersion) (blobstore.BlobInfo, error)

StatWithStorageFormat looks up disk metadata for the blob file with the given storage format version. This avoids the potential need to check multiple storage formats for the blob when the format is already known.

func (*BadBlobs) Trash 

func (bad *BadBlobs) Trash(ctx context.Context, ref blobstore.BlobRef, timestamp time.Time) error

Trash deletes the blob with the namespace and key.

func (*BadBlobs) TrashWithStorageFormat added in v1.109.1 

func (bad *BadBlobs) TrashWithStorageFormat(ctx context.Context, ref blobstore.BlobRef, formatVer blobstore.FormatVersion, timestamp time.Time) error

TrashWithStorageFormat moves into trash the blob with the namespace, key and specific storage format.

func (*BadBlobs) TryRestoreTrashBlob added in v1.99.1 

func (bad *BadBlobs) TryRestoreTrashBlob(ctx context.Context, ref blobstore.BlobRef) error

TryRestoreTrashBlob attempts to restore a blob from the trash.

func (*BadBlobs) VerifyStorageDir 

func (bad *BadBlobs) VerifyStorageDir(ctx context.Context, id storj.NodeID) error

VerifyStorageDir verifies that the storage directory is correct by checking for the existence and validity of the verification file.

func (*BadBlobs) WalkNamespace 

func (bad *BadBlobs) WalkNamespace(ctx context.Context, namespace []byte, startFromPrefix string, walkFunc func(blobstore.BlobInfo) error) error

WalkNamespace executes walkFunc for each locally stored blob in the given namespace. If walkFunc returns a non-nil error, WalkNamespace will stop iterating and return the error immediately.

type BadDB 

type BadDB struct {
	storagenode.DB
	Blobs ErrorBlobs
	// contains filtered or unexported fields
}

BadDB implements bad storage node DB.

func NewBadDB 

func NewBadDB(log *zap.Logger, db storagenode.DB) *BadDB

NewBadDB creates a new bad storage node DB. Use SetError to manually configure the error returned by all blob operations.

func (*BadDB) Pieces 

func (bad *BadDB) Pieces() blobstore.Blobs

Pieces returns the blob store.

func (*BadDB) SetCheckError 

func (bad *BadDB) SetCheckError(err error)

SetCheckError sets an error to be returned for check and verification operations.

func (*BadDB) SetError 

func (bad *BadDB) SetError(err error)

SetError sets an error to be returned for blob operations.

type ErrorBlobs 

type ErrorBlobs interface {
	blobstore.Blobs
	SetError(err error)
	SetCheckError(err error)
}

ErrorBlobs is the interface of blobstore.Blobs with the SetError method added. This allows the BadDB{}.Blobs member to be replaced with something that has specific behavior changes.

type LimitedSpaceBlobs 

type LimitedSpaceBlobs struct {
	blobstore.Blobs
	// contains filtered or unexported fields
}

LimitedSpaceBlobs implements a limited space blob store.

func (*LimitedSpaceBlobs) DiskInfo added in v1.98.1 

func (limspace *LimitedSpaceBlobs) DiskInfo(ctx context.Context) (blobstore.DiskInfo, error)

DiskInfo returns the disk info.

func (*LimitedSpaceBlobs) FreeSpace 

func (limspace *LimitedSpaceBlobs) FreeSpace(ctx context.Context) (int64, error)

FreeSpace returns how much free space left for writing.

type SlowBlobs 

type SlowBlobs struct {
	// contains filtered or unexported fields
}

SlowBlobs implements a slow blob store.

func (*SlowBlobs) CheckWritability 

func (slow *SlowBlobs) CheckWritability(ctx context.Context) error

CheckWritability tests writability of the storage directory by creating and deleting a file.

func (*SlowBlobs) Close 

func (slow *SlowBlobs) Close() error

Close closes the blob store and any resources associated with it.

func (*SlowBlobs) Create 

func (slow *SlowBlobs) Create(ctx context.Context, ref blobstore.BlobRef) (blobstore.BlobWriter, error)

Create creates a new blob that can be written.

func (*SlowBlobs) CreateVerificationFile 

func (slow *SlowBlobs) CreateVerificationFile(ctx context.Context, id storj.NodeID) error

CreateVerificationFile creates a file to be used for storage directory verification.

func (*SlowBlobs) Delete 

func (slow *SlowBlobs) Delete(ctx context.Context, ref blobstore.BlobRef) error

Delete deletes the blob with the namespace and key.

func (*SlowBlobs) DeleteNamespace 

func (slow *SlowBlobs) DeleteNamespace(ctx context.Context, ref []byte) (err error)

DeleteNamespace deletes blobs of specific satellite, used after successful GE only.

func (*SlowBlobs) DeleteTrashNamespace added in v1.88.2 

func (slow *SlowBlobs) DeleteTrashNamespace(ctx context.Context, namespace []byte) error

DeleteTrashNamespace deletes the trash folder for the specified namespace.

func (*SlowBlobs) DeleteWithStorageFormat 

func (slow *SlowBlobs) DeleteWithStorageFormat(ctx context.Context, ref blobstore.BlobRef, formatVer blobstore.FormatVersion) error

DeleteWithStorageFormat deletes the blob with the namespace, key, and format version.

func (*SlowBlobs) DiskInfo added in v1.98.1 

func (slow *SlowBlobs) DiskInfo(ctx context.Context) (blobstore.DiskInfo, error)

DiskInfo returns the disk space information.

func (*SlowBlobs) EmptyTrash 

func (slow *SlowBlobs) EmptyTrash(ctx context.Context, namespace []byte, trashedBefore time.Time) (int64, [][]byte, error)

EmptyTrash empties the trash.

func (*SlowBlobs) ListNamespaces 

func (slow *SlowBlobs) ListNamespaces(ctx context.Context) ([][]byte, error)

ListNamespaces returns all namespaces that might be storing data.

func (*SlowBlobs) Open 

func (slow *SlowBlobs) Open(ctx context.Context, ref blobstore.BlobRef) (blobstore.BlobReader, error)

Open opens a reader with the specified namespace and key.

func (*SlowBlobs) OpenWithStorageFormat 

func (slow *SlowBlobs) OpenWithStorageFormat(ctx context.Context, ref blobstore.BlobRef, formatVer blobstore.FormatVersion) (blobstore.BlobReader, error)

OpenWithStorageFormat opens a reader for the already-located blob, avoiding the potential need to check multiple storage formats to find the blob.

func (*SlowBlobs) RestoreTrash 

func (slow *SlowBlobs) RestoreTrash(ctx context.Context, namespace []byte) ([][]byte, error)

RestoreTrash restores all files in the trash.

func (*SlowBlobs) SetLatency 

func (slow *SlowBlobs) SetLatency(delay time.Duration)

SetLatency configures the blob store to sleep for delay duration for all operations. A zero or negative delay means no sleep.

func (*SlowBlobs) SpaceUsedForBlobs 

func (slow *SlowBlobs) SpaceUsedForBlobs(ctx context.Context) (int64, error)

SpaceUsedForBlobs adds up how much is used in all namespaces.

func (*SlowBlobs) SpaceUsedForBlobsInNamespace 

func (slow *SlowBlobs) SpaceUsedForBlobsInNamespace(ctx context.Context, namespace []byte) (int64, error)

SpaceUsedForBlobsInNamespace adds up how much is used in the given namespace.

func (*SlowBlobs) SpaceUsedForTrash 

func (slow *SlowBlobs) SpaceUsedForTrash(ctx context.Context) (int64, error)

SpaceUsedForTrash adds up how much is used in all namespaces.

func (*SlowBlobs) Stat 

func (slow *SlowBlobs) Stat(ctx context.Context, ref blobstore.BlobRef) (blobstore.BlobInfo, error)

Stat looks up disk metadata on the blob file.

func (*SlowBlobs) StatWithStorageFormat 

func (slow *SlowBlobs) StatWithStorageFormat(ctx context.Context, ref blobstore.BlobRef, formatVer blobstore.FormatVersion) (blobstore.BlobInfo, error)

StatWithStorageFormat looks up disk metadata for the blob file with the given storage format version. This avoids the potential need to check multiple storage formats for the blob when the format is already known.

func (*SlowBlobs) Trash 

func (slow *SlowBlobs) Trash(ctx context.Context, ref blobstore.BlobRef, timestamp time.Time) error

Trash deletes the blob with the namespace and key.

func (*SlowBlobs) TrashWithStorageFormat added in v1.109.1 

func (slow *SlowBlobs) TrashWithStorageFormat(ctx context.Context, ref blobstore.BlobRef, formatVer blobstore.FormatVersion, timestamp time.Time) error

TrashWithStorageFormat moves into trash the blob with the namespace, key and specific storage format.

func (*SlowBlobs) TryRestoreTrashBlob added in v1.99.1 

func (slow *SlowBlobs) TryRestoreTrashBlob(ctx context.Context, ref blobstore.BlobRef) error

TryRestoreTrashBlob attempts to restore a blob from trash.

func (*SlowBlobs) VerifyStorageDir 

func (slow *SlowBlobs) VerifyStorageDir(ctx context.Context, id storj.NodeID) error

VerifyStorageDir verifies that the storage directory is correct by checking for the existence and validity of the verification file.

func (*SlowBlobs) WalkNamespace 

func (slow *SlowBlobs) WalkNamespace(ctx context.Context, namespace []byte, startFromPrefix string, walkFunc func(blobstore.BlobInfo) error) error

WalkNamespace executes walkFunc for each locally stored blob in the given namespace. If walkFunc returns a non-nil error, WalkNamespace will stop iterating and return the error immediately.

type SlowDB 

type SlowDB struct {
	storagenode.DB
	// contains filtered or unexported fields
}

SlowDB implements slow storage node DB.

func NewSlowDB 

func NewSlowDB(log *zap.Logger, db storagenode.DB) *SlowDB

NewSlowDB creates a new slow storage node DB wrapping the provided db. Use SetLatency to dynamically configure the latency of all blob operations.

func (*SlowDB) Pieces 

func (slow *SlowDB) Pieces() blobstore.Blobs

Pieces returns the blob store.

func (*SlowDB) SetLatency 

func (slow *SlowDB) SetLatency(delay time.Duration)

SetLatency enables a sleep for delay duration for all blob operations. A zero or negative delay means no sleep.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.