module

package
v0.2.3-alpha.17 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 30, 2023 License: GPL-3.0 Imports: 14 Imported by: 0

README

Module

The Module is a self-contained logical component of SP, with necessary interactions between modules handled by the GfSp framework. The implementation of the module can be customized as needed. For instance, while the GfSp framework requires object approval before uploading, SP can customize whether to agree with the approval.

Concept

Front Modules

The Front Modules are responsible for handling user requests. The Gater generates corresponding tasks and sends them to the Front Modules. The Front Modules verify the correctness of the request and perform additional tasks after handling the request. To accomplish this, the Front Modules have three interfaces for each task type: PreHandleXXXTask, HandleXXXTask and PostHandleXXXTask. The Front Modules consist of Approver, Downloader and Uploader.

Background Modules

The Background Modules are responsible for handling internal tasks of SP, which are generated internally and thus have guaranteed information correctness. As a result, there is only one interface HandleXXXTask for these tasks. The Background Modules consist of Authenticator, TaskExecutor, Manager, P2P, Receiver and Signer.

Module Type

The GfSp framework comprises several modules, including Gater, Approver, Authenticator, Uploader, Downloader, Manager, P2P, TaskExecutor, Receiver, Signer, Metadata and BlockSyncer. Additionally, the GfSp framework supports the extension of customized modules as required. Once registered in the GfSp framework and executing the modular interface, these customized modules will be initialized and scheduled.

Gater

Gater module serves as the gateway for SP, providing HTTP services and adhering to the S3 protocol. It generates tasks corresponding to user requests and forwards them to other modules within SP. Since Gater does not allow customization, no interface is defined in the modular file.

Authenticator

Authenticator module is responsible for verifying authentication.

Approver

Approver module is responsible for handling approval requests, specifically CreateBucketApproval and CreateObjectApproval.

Uploader

Uploader module handles the put object requests from user accounts and stores payload data into piece store of the primary SP.

Downloader

Downloader module is responsible for handling get object request from user account and get challenge info request from other components in the Greenfield system.

TaskExecutor

TaskExecutor module is responsible for handling background task. This module can request tasks from the Manager module, execute them and report the results or status back to the Manager. The tasks it can handle include ReplicatePieceTask, SealObjectTask, ReceivePieceTask, GCObjectTask, GCZombiePieceTask, and GCMetaTask.

Manager

Manager module is responsible for managing task scheduling of SP and other management functions.

P2P

P2P module is responsible for handling the interaction of control information between SPs. It handles ask replicate piece approval requests by broadcasting the approval to other SPs, waiting for responses and returning the approvals if the minimum or maximum approved number is reached before the timeout.

Receiver

Receiver module receives data from the primary SP, calculates the integrity hash of the data, signs it, and returns it to the primary SP for sealing on a greenfield.

Signer

Signer module handles the signing of the SP data on the Greenfield chain operator and holds all of the SP's private keys. Due to the sequence number of the SP account, it must be a singleton.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ApprovalModularName              = strings.ToLower("Approver")
	ApprovalModularDescription       = "Handles the ask crate bucket/object and replicates piece approval request."
	AuthenticationModularName        = strings.ToLower("Authenticator")
	AuthenticationModularDescription = "Checks authentication."
	DownloadModularName              = strings.ToLower("Downloader")
	DownloadModularDescription       = "Downloads object and gets challenge info and statistical read traffic from the backend."
	ExecuteModularName               = strings.ToLower("TaskExecutor")
	ExecuteModularDescription        = "Executes background tasks."
	GateModularName                  = strings.ToLower("Gateway")
	GateModularDescription           = "Receives the user request and routes to the responding service."
	ManageModularName                = strings.ToLower("Manager")
	ManageModularDescription         = "Manages SPs and schedules tasks."
	P2PModularName                   = strings.ToLower("p2p")
	P2PModularDescription            = "Communicates between SPs on p2p protocol."
	ReceiveModularName               = strings.ToLower("Receiver")
	ReceiveModularDescription        = "Receives data pieces of an object from other storage provider and store."
	SignModularName                  = strings.ToLower("Signer")
	SignModularDescription           = "Signs the transaction and broadcasts to chain."
	UploadModularName                = strings.ToLower("Uploader")
	UploadModularDescription         = "Uploads object payload to primary SP."
	BlockSyncerModularName           = strings.ToLower("BlockSyncer")
	BlockSyncerModularDescription    = "Synchronize data on the chain to SP"
	MetadataModularName              = strings.ToLower("Metadata")
	MetadataModularDescription       = "Retrieves sp metadata and info."
)
View Source 
var (
	ErrNilModular = errors.New("call nil module, please check again")
)

Functions

This section is empty.

Types

type Approver 

type Approver interface {
	Modular
	// PreCreateBucketApproval prepares to handle CreateBucketApproval, it can do some
	// checks such as checking for duplicates, if limitation of SP has been reached, etc.
	PreCreateBucketApproval(ctx context.Context, task task.ApprovalCreateBucketTask) error
	// HandleCreateBucketApprovalTask handles the CreateBucketApproval, it can set expired height, sign the MsgCreateBucket and so on.
	HandleCreateBucketApprovalTask(ctx context.Context, task task.ApprovalCreateBucketTask) (bool, error)
	// PostCreateBucketApproval is called after HandleCreateBucketApprovalTask, it can recycle resources, make statistics
	// and do some other operations.
	PostCreateBucketApproval(ctx context.Context, task task.ApprovalCreateBucketTask)

	// PreMigrateBucketApproval prepares to handle MigrateBucketApproval, it can do some
	// checks such as checking for duplicates, if limitation of SP has been reached, etc.
	PreMigrateBucketApproval(ctx context.Context, task task.ApprovalMigrateBucketTask) error
	// HandleMigrateBucketApprovalTask handles the MigrateBucketApproval, it can set expired height, sign the MsgMigrateBucket and so on.
	HandleMigrateBucketApprovalTask(ctx context.Context, task task.ApprovalMigrateBucketTask) (bool, error)
	// PostMigrateBucketApproval is called after HandleMigrateBucketApprovalTask, it can recycle resources, make statistics
	// and do some other operations.
	PostMigrateBucketApproval(ctx context.Context, task task.ApprovalMigrateBucketTask)

	// PreCreateObjectApproval prepares to handle CreateObjectApproval, it can do some
	// checks such as check for duplicates, if limitation of SP has been reached, etc.
	PreCreateObjectApproval(ctx context.Context, task task.ApprovalCreateObjectTask) error
	// HandleCreateObjectApprovalTask handles the CreateObjectApproval, it can set expired height, sign the MsgCreateObject and so on.
	HandleCreateObjectApprovalTask(ctx context.Context, task task.ApprovalCreateObjectTask) (bool, error)
	// PostCreateObjectApproval is called after HandleCreateObjectApprovalTask, it can
	// recycle resources, make statistics and do some other operations.
	PostCreateObjectApproval(ctx context.Context, task task.ApprovalCreateObjectTask)
	// QueryTasks queries tasks that running on approver by task sub-key.
	QueryTasks(ctx context.Context, subKey task.TKey) ([]task.Task, error)
}

Approver is an abstract interface to handle asking approval requests.

type AuthOpType 

type AuthOpType int32

AuthOpType defines the operator type used to authentication verification. 

const (
	// AuthOpTypeUnKnown defines the default value of AuthOpType
	AuthOpTypeUnKnown AuthOpType = iota
	// AuthOpAskCreateBucketApproval defines the AskCreateBucketApproval operator
	AuthOpAskCreateBucketApproval
	// AuthOpAskMigrateBucketApproval defines the AskMigrateBucketApproval operator
	AuthOpAskMigrateBucketApproval
	// AuthOpAskCreateObjectApproval defines the AskCreateObjectApproval operator
	AuthOpAskCreateObjectApproval
	// AuthOpTypeGetChallengePieceInfo defines the GetChallengePieceInfo operator
	AuthOpTypeGetChallengePieceInfo
	// AuthOpTypePutObject defines the PutObject operator
	AuthOpTypePutObject
	// AuthOpTypeGetObject defines the GetObject operator
	AuthOpTypeGetObject
	// AuthOpTypeGetUploadingState defines the GetUploadingState operator
	AuthOpTypeGetUploadingState
	// AuthOpTypeGetBucketQuota defines the GetBucketQuota operator
	AuthOpTypeGetBucketQuota
	// AuthOpTypeListBucketReadRecord defines the ListBucketReadRecord operator
	AuthOpTypeListBucketReadRecord
	// AuthOpTypeGetRecoveryPiece defines the GetRecoveryPiece operator
	AuthOpTypeGetRecoveryPiece
)

type Authenticator 

type Authenticator interface {
	Modular
	// VerifyAuthentication verifies the operator authentication.
	VerifyAuthentication(ctx context.Context, auth AuthOpType, account, bucket, object string) (bool, error)
	// GetAuthNonce get the auth nonce for which the dApp or client can generate EDDSA key pairs.
	GetAuthNonce(ctx context.Context, account string, domain string) (*spdb.OffChainAuthKey, error)
	// UpdateUserPublicKey updates the user public key once the dApp or client generates the EDDSA key pairs.
	UpdateUserPublicKey(ctx context.Context, account string, domain string, currentNonce int32, nonce int32,
		userPublicKey string, expiryDate int64) (bool, error)
	// VerifyOffChainSignature verifies the signature signed by user's EDDSA private key.
	VerifyOffChainSignature(ctx context.Context, account string, domain string, offChainSig string, realMsgToSign string) (bool, error)
}

Authenticator is an abstract interface to verify users authentication.

type Downloader 

type Downloader interface {
	Modular
	// PreDownloadObject prepares to handle DownloadObject, it can do some checks
	// such as checking for duplicates, if limitation of SP has been reached, etc.
	PreDownloadObject(ctx context.Context, task task.DownloadObjectTask) error
	// HandleDownloadObjectTask handles the DownloadObject and get data from piece store.
	HandleDownloadObjectTask(ctx context.Context, task task.DownloadObjectTask) ([]byte, error)
	// PostDownloadObject is called after HandleDownloadObjectTask, it can recycle
	// resources, make statistics and do some other operations..
	PostDownloadObject(ctx context.Context, task task.DownloadObjectTask)

	// PreDownloadPiece prepares to handle DownloadPiece, it can do some checks such as check for duplicates,
	// if limitation of SP has been reached, etc.
	PreDownloadPiece(ctx context.Context, task task.DownloadPieceTask) error
	// HandleDownloadPieceTask handles the DownloadPiece and get data from piece store.
	HandleDownloadPieceTask(ctx context.Context, task task.DownloadPieceTask) ([]byte, error)
	// PostDownloadPiece is called after HandleDownloadPieceTask, it can recycle
	// resources, make statistics and do some other operations.
	PostDownloadPiece(ctx context.Context, task task.DownloadPieceTask)

	// PreChallengePiece prepares to handle ChallengePiece, it can do some checks
	// such as checking for duplicates, if limitation of SP has been reached, etc.
	PreChallengePiece(ctx context.Context, task task.ChallengePieceTask) error
	// HandleChallengePiece handles ChallengePiece, get piece data from piece store and get integrity hash from db.
	HandleChallengePiece(ctx context.Context, task task.ChallengePieceTask) ([]byte, [][]byte, []byte, error)
	// PostChallengePiece is called after HandleChallengePiece, it can recycle resources, make statistics
	// and do some other operations.
	PostChallengePiece(ctx context.Context, task task.ChallengePieceTask)
	// QueryTasks queries download/challenge tasks that running on downloader by task sub-key.
	QueryTasks(ctx context.Context, subKey task.TKey) ([]task.Task, error)
}

Downloader is an abstract interface to handle getting object requests from users' account, and getting challenge info requests from other components in the system.

type Manager 

type Manager interface {
	Modular
	// DispatchTask dispatches the task to TaskExecutor module when it asks tasks.
	// It will consider task remaining resources when dispatching task.
	DispatchTask(ctx context.Context, limit rcmgr.Limit) (task.Task, error)
	// QueryTasks queries tasks that hold on manager by task sub-key.
	QueryTasks(ctx context.Context, subKey task.TKey) ([]task.Task, error)
	// QueryBucketMigrate queries tasks that hold on manager by task sub-key.
	QueryBucketMigrate(ctx context.Context) (*gfspserver.GfSpQueryBucketMigrateResponse, error)
	// QuerySpExit queries tasks that hold on manager by task sub-key.
	QuerySpExit(ctx context.Context) (*gfspserver.GfSpQuerySpExitResponse, error)
	// HandleCreateUploadObjectTask handles the CreateUploadObject request from Uploader, before Uploader handles
	// the users' UploadObject requests, it should send CreateUploadObject requests to Manager ask if it's ok.
	// Through this interface SP implements the global uploading object strategy.
	//
	// For example: control the concurrency of global uploads, avoid repeated uploads, rate control, etc.
	HandleCreateUploadObjectTask(ctx context.Context, task task.UploadObjectTask) error
	// HandleDoneUploadObjectTask handles the result of uploading object payload data to primary, Manager should
	// generate ReplicatePieceTask for TaskExecutor to run.
	HandleDoneUploadObjectTask(ctx context.Context, task task.UploadObjectTask) error
	// HandleCreateResumableUploadObjectTask handles the CreateUploadObject request from
	// Uploader, before Uploader handles the user's UploadObject request, it should
	// send CreateUploadObject request to Manager ask if it's ok. Through this
	// interface that SP implements the global upload object strategy.
	//
	HandleCreateResumableUploadObjectTask(ctx context.Context, task task.ResumableUploadObjectTask) error

	// HandleDoneResumableUploadObjectTask handles the result of resumable uploading object payload data to primary,
	// Manager should generate ReplicatePieceTask for TaskExecutor to run.
	HandleDoneResumableUploadObjectTask(ctx context.Context, task task.ResumableUploadObjectTask) error
	// HandleReplicatePieceTask handles the result of replicating piece data to secondary SPs,
	// the request comes from TaskExecutor.
	HandleReplicatePieceTask(ctx context.Context, task task.ReplicatePieceTask) error
	// HandleSealObjectTask handles the result of sealing object to the greenfield the request comes from TaskExecutor.
	HandleSealObjectTask(ctx context.Context, task task.SealObjectTask) error
	// HandleReceivePieceTask handles the result of receiving piece task, the request comes from Receiver that
	// reports have completed ReceivePieceTask to manager and TaskExecutor that the result of confirming whether
	// the object that is synced to secondary SP has been sealed.
	HandleReceivePieceTask(ctx context.Context, task task.ReceivePieceTask) error
	// HandleGCObjectTask handles GCObjectTask, the request comes from TaskExecutor.
	HandleGCObjectTask(ctx context.Context, task task.GCObjectTask) error
	// HandleGCZombiePieceTask handles GCZombiePieceTask, the request comes from TaskExecutor.
	HandleGCZombiePieceTask(ctx context.Context, task task.GCZombiePieceTask) error
	// HandleGCMetaTask handles GCMetaTask, the request comes from TaskExecutor.
	HandleGCMetaTask(ctx context.Context, task task.GCMetaTask) error
	// HandleDownloadObjectTask handles DownloadObjectTask, the request comes from Downloader.
	HandleDownloadObjectTask(ctx context.Context, task task.DownloadObjectTask) error
	// HandleChallengePieceTask handles ChallengePieceTask, the request comes from Downloader.
	HandleChallengePieceTask(ctx context.Context, task task.ChallengePieceTask) error
	// PickVirtualGroupFamily is used to pick vgf for the new bucket.
	PickVirtualGroupFamily(ctx context.Context, task task.ApprovalCreateBucketTask) (uint32, error)
	// HandleRecoverPieceTask handles the result of recovering piece task, the request comes from TaskExecutor.
	HandleRecoverPieceTask(ctx context.Context, task task.RecoveryPieceTask) error
	// NotifyMigrateSwapOut is used to notify dest sp migrate swap out.
	NotifyMigrateSwapOut(ctx context.Context, swapOut *virtualgrouptypes.MsgSwapOut) error
	// HandleMigrateGVGTask handles MigrateGVGTask, the request from TaskExecutor.
	HandleMigrateGVGTask(ctx context.Context, task task.MigrateGVGTask) error
}

Manager is an abstract interface to do some internal services management, it is responsible for task scheduling and other management of SP.

type Modular 

type Modular interface {
	lifecycle.Service
	// ReserveResource reserves the resources from Modular resources pool.
	ReserveResource(ctx context.Context, state *rcmgr.ScopeStat) (rcmgr.ResourceScopeSpan, error)
	// ReleaseResource releases the resources to Modular resources pool.
	ReleaseResource(ctx context.Context, scope rcmgr.ResourceScopeSpan)
}

Modular is a common interface for submodules that are scheduled by the GfSp framework. It inherits lifecycle.Service interface, which is used to manage lifecycle of services. Additionally, Modular is managed by ResourceManager, which allows the GfSp framework to reserve and release resources from the Modular resource pool.

type NilModular 

type NilModular struct{}

func (*NilModular) AskTask 

func (*NilModular) AskTask(context.Context) error

func (*NilModular) CompleteMigrateBucket added in v0.2.3 

func (*NilModular) CompleteMigrateBucket(ctx context.Context, migrateBucket *storagetypes.MsgCompleteMigrateBucket) (string, error)

func (*NilModular) CompleteSPExit added in v0.2.3 

func (*NilModular) CompleteSPExit(ctx context.Context, completeSPExit *virtualgrouptypes.MsgCompleteStorageProviderExit) (string, error)

func (*NilModular) CompleteSwapOut added in v0.2.3 

func (*NilModular) CompleteSwapOut(ctx context.Context, completeSwapOut *virtualgrouptypes.MsgCompleteSwapOut) (string, error)

func (*NilModular) CreateGlobalVirtualGroup added in v0.2.3 

func (*NilModular) CreateGlobalVirtualGroup(context.Context, *virtualgrouptypes.MsgCreateGlobalVirtualGroup) error

func (*NilModular) DiscontinueBucket 

func (*NilModular) DiscontinueBucket(context.Context, *storagetypes.MsgDiscontinueBucket) (string, error)

func (*NilModular) HandleChallengePiece 

func (*NilModular) HandleChallengePiece(context.Context, task.ChallengePieceTask) ([]byte, [][]byte, []byte, error)

func (*NilModular) HandleDownloadObjectTask 

func (*NilModular) HandleDownloadObjectTask(context.Context, task.DownloadObjectTask) ([]byte, error)

func (*NilModular) HandleDownloadPieceTask 

func (*NilModular) HandleDownloadPieceTask(context.Context, task.DownloadPieceTask) ([]byte, error)

func (*NilModular) HandleGCMetaTask 

func (*NilModular) HandleGCMetaTask(context.Context, task.GCMetaTask)

func (*NilModular) HandleGCObjectTask 

func (*NilModular) HandleGCObjectTask(context.Context, task.GCObjectTask)

func (*NilModular) HandleGCZombiePieceTask 

func (*NilModular) HandleGCZombiePieceTask(context.Context, task.GCZombiePieceTask)

func (*NilModular) HandleMigrateGVGTask added in v0.2.3 

func (*NilModular) HandleMigrateGVGTask(ctx context.Context, gvgTask task.MigrateGVGTask)

func (*NilModular) HandleQueryBootstrap 

func (*NilModular) HandleQueryBootstrap(context.Context) ([]string, error)

func (*NilModular) HandleReceivePieceTask 

func (*NilModular) HandleReceivePieceTask(context.Context, task.ReceivePieceTask)

func (*NilModular) HandleReplicatePieceApproval 

func (*NilModular) HandleReplicatePieceApproval(context.Context, task.ApprovalReplicatePieceTask, int32, int32, int64) ([]task.ApprovalReplicatePieceTask, error)

func (*NilModular) HandleReplicatePieceTask 

func (*NilModular) HandleReplicatePieceTask(context.Context, task.ReplicatePieceTask)

func (*NilModular) HandleSealObjectTask 

func (*NilModular) HandleSealObjectTask(context.Context, task.SealObjectTask)

func (*NilModular) Name 

func (*NilModular) Name() string

func (*NilModular) PostChallengePiece 

func (*NilModular) PostChallengePiece(context.Context, task.ChallengePieceTask)

func (*NilModular) PostDownloadObject 

func (*NilModular) PostDownloadObject(context.Context, task.DownloadObjectTask)

func (*NilModular) PostDownloadPiece 

func (*NilModular) PostDownloadPiece(context.Context, task.DownloadPieceTask)

func (*NilModular) PreChallengePiece 

func (*NilModular) PreChallengePiece(context.Context, task.ChallengePieceTask) error

func (*NilModular) PreDownloadObject 

func (*NilModular) PreDownloadObject(context.Context, task.DownloadObjectTask) error

func (*NilModular) PreDownloadPiece 

func (*NilModular) PreDownloadPiece(context.Context, task.DownloadPieceTask) error

func (*NilModular) QueryTasks 

func (*NilModular) QueryTasks(ctx context.Context, keyPrefix task.TKey) ([]task.Task, error)

func (*NilModular) RejectUnSealObject 

func (*NilModular) RejectUnSealObject(context.Context, *storagetypes.MsgRejectSealObject) (string, error)

func (*NilModular) ReleaseResource 

func (*NilModular) ReleaseResource(context.Context, rcmgr.ResourceScopeSpan)

func (*NilModular) ReportTask 

func (*NilModular) ReportTask(context.Context, task.Task) error

func (*NilModular) ReserveResource 

func (*NilModular) ReserveResource(context.Context, *rcmgr.ScopeStat) (rcmgr.ResourceScopeSpan, error)

func (*NilModular) SPExit added in v0.2.3 

func (*NilModular) SPExit(ctx context.Context, spExit *virtualgrouptypes.MsgStorageProviderExit) (string, error)

func (*NilModular) SealObject 

func (*NilModular) SealObject(context.Context, *storagetypes.MsgSealObject) (string, error)

func (*NilModular) SignCreateBucketApproval 

func (*NilModular) SignCreateBucketApproval(context.Context, *storagetypes.MsgCreateBucket) ([]byte, error)

func (*NilModular) SignCreateObjectApproval 

func (*NilModular) SignCreateObjectApproval(context.Context, *storagetypes.MsgCreateObject) ([]byte, error)

func (*NilModular) SignMigrateBucketApproval added in v0.2.3 

func (*NilModular) SignMigrateBucketApproval(context.Context, *storagetypes.MsgMigrateBucket) ([]byte, error)

func (*NilModular) SignMigratePiece added in v0.2.3 

func (*NilModular) SignMigratePiece(ctx context.Context, task *gfsptask.GfSpMigratePieceTask) ([]byte, error)

func (*NilModular) SignP2PPingMsg 

func (*NilModular) SignP2PPingMsg(context.Context, *gfspp2p.GfSpPing) ([]byte, error)

func (*NilModular) SignP2PPongMsg 

func (*NilModular) SignP2PPongMsg(context.Context, *gfspp2p.GfSpPong) ([]byte, error)

func (*NilModular) SignReceivePieceTask 

func (*NilModular) SignReceivePieceTask(context.Context, task.ReceivePieceTask) ([]byte, error)

func (*NilModular) SignRecoveryPieceTask added in v0.2.3 

func (*NilModular) SignRecoveryPieceTask(context.Context, task.RecoveryPieceTask) ([]byte, error)

func (*NilModular) SignReplicatePieceApproval 

func (*NilModular) SignReplicatePieceApproval(context.Context, task.ApprovalReplicatePieceTask) ([]byte, error)

func (*NilModular) SignSecondarySPMigrationBucket added in v0.2.3 

func (*NilModular) SignSecondarySPMigrationBucket(ctx context.Context, signDoc *storagetypes.SecondarySpMigrationBucketSignDoc) ([]byte, error)

func (*NilModular) SignSecondarySealBls added in v0.2.3 

func (*NilModular) SignSecondarySealBls(context.Context, uint64, uint32, [][]byte) ([]byte, error)

func (*NilModular) SignSwapOut added in v0.2.3 

func (*NilModular) SignSwapOut(ctx context.Context, swapOut *virtualgrouptypes.MsgSwapOut) ([]byte, error)

func (*NilModular) Start 

func (*NilModular) Start(context.Context) error

func (*NilModular) Stop 

func (*NilModular) Stop(context.Context) error

func (*NilModular) SwapOut added in v0.2.3 

func (*NilModular) SwapOut(ctx context.Context, swapOut *virtualgrouptypes.MsgSwapOut) (string, error)

func (*NilModular) UpdateSPPrice added in v0.2.3 

func (*NilModular) UpdateSPPrice(ctx context.Context, price *sptypes.MsgUpdateSpStoragePrice) (string, error)

type NullModular 

type NullModular struct{}

func (*NullModular) DispatchTask 

func (*NullModular) DispatchTask(context.Context, rcmgr.Limit) (task.Task, error)

func (*NullModular) GetAuthNonce 

func (*NullModular) GetAuthNonce(ctx context.Context, account string, domain string) (*corespdb.OffChainAuthKey, error)

func (*NullModular) HandleChallengePieceTask 

func (*NullModular) HandleChallengePieceTask(context.Context, task.ChallengePieceTask) error

func (*NullModular) HandleCreateBucketApprovalTask 

func (*NullModular) HandleCreateBucketApprovalTask(context.Context, task.ApprovalCreateBucketTask) (bool, error)

func (*NullModular) HandleCreateObjectApprovalTask 

func (*NullModular) HandleCreateObjectApprovalTask(context.Context, task.ApprovalCreateObjectTask) (bool, error)

func (*NullModular) HandleCreateResumableUploadObjectTask added in v0.2.3 

func (*NullModular) HandleCreateResumableUploadObjectTask(context.Context, task.ResumableUploadObjectTask) error

func (*NullModular) HandleCreateUploadObjectTask 

func (*NullModular) HandleCreateUploadObjectTask(context.Context, task.UploadObjectTask) error

func (*NullModular) HandleDoneResumableUploadObjectTask added in v0.2.3 

func (*NullModular) HandleDoneResumableUploadObjectTask(context.Context, task.ResumableUploadObjectTask) error

func (*NullModular) HandleDoneUploadObjectTask 

func (*NullModular) HandleDoneUploadObjectTask(context.Context, task.UploadObjectTask) error

func (*NullModular) HandleDownloadObjectTask 

func (*NullModular) HandleDownloadObjectTask(context.Context, task.DownloadObjectTask) error

func (*NullModular) HandleGCMetaTask 

func (*NullModular) HandleGCMetaTask(context.Context, task.GCMetaTask) error

func (*NullModular) HandleGCObjectTask 

func (*NullModular) HandleGCObjectTask(context.Context, task.GCObjectTask) error

func (*NullModular) HandleGCZombiePieceTask 

func (*NullModular) HandleGCZombiePieceTask(context.Context, task.GCZombiePieceTask) error

func (*NullModular) HandleMigrateBucketApprovalTask added in v0.2.3 

func (*NullModular) HandleMigrateBucketApprovalTask(context.Context, task.ApprovalMigrateBucketTask) (bool, error)

func (*NullModular) HandleMigrateGVGTask added in v0.2.3 

func (*NullModular) HandleMigrateGVGTask(ctx context.Context, gvgTask task.MigrateGVGTask) error

func (*NullModular) HandleReceivePieceTask 

func (*NullModular) HandleReceivePieceTask(context.Context, task.ReceivePieceTask) error

func (*NullModular) HandleRecoverPieceTask added in v0.2.3 

func (*NullModular) HandleRecoverPieceTask(ctx context.Context, task task.RecoveryPieceTask) error

func (*NullModular) HandleReplicatePieceApproval 

func (*NullModular) HandleReplicatePieceApproval(context.Context, task.ApprovalReplicatePieceTask) (bool, error)

func (*NullModular) HandleReplicatePieceTask 

func (*NullModular) HandleReplicatePieceTask(context.Context, task.ReplicatePieceTask) error

func (*NullModular) HandleResumableUploadObjectTask added in v0.2.3 

func (*NullModular) HandleResumableUploadObjectTask(ctx context.Context, task task.ResumableUploadObjectTask, stream io.Reader) error

func (*NullModular) HandleSealObjectTask 

func (*NullModular) HandleSealObjectTask(context.Context, task.SealObjectTask) error

func (*NullModular) HandleUploadObjectTask 

func (*NullModular) HandleUploadObjectTask(ctx context.Context, task task.UploadObjectTask, stream io.Reader) error

func (*NullModular) Name 

func (*NullModular) Name() string

func (*NullModular) NotifyMigrateSwapOut added in v0.2.3 

func (*NullModular) NotifyMigrateSwapOut(context.Context, *virtualgrouptypes.MsgSwapOut) error

func (*NullModular) PickVirtualGroupFamily added in v0.2.3 

func (*NullModular) PickVirtualGroupFamily(context.Context, task.ApprovalCreateBucketTask) (uint32, error)

func (*NullModular) PostCreateBucketApproval 

func (*NullModular) PostCreateBucketApproval(context.Context, task.ApprovalCreateBucketTask)

func (*NullModular) PostCreateObjectApproval 

func (*NullModular) PostCreateObjectApproval(context.Context, task.ApprovalCreateObjectTask)

func (*NullModular) PostMigrateBucketApproval added in v0.2.3 

func (*NullModular) PostMigrateBucketApproval(context.Context, task.ApprovalMigrateBucketTask)

func (*NullModular) PostReplicatePieceApproval 

func (*NullModular) PostReplicatePieceApproval(context.Context, task.ApprovalReplicatePieceTask)

func (*NullModular) PostResumableUploadObject added in v0.2.3 

func (*NullModular) PostResumableUploadObject(ctx context.Context, task task.ResumableUploadObjectTask)

func (*NullModular) PostUploadObject 

func (*NullModular) PostUploadObject(ctx context.Context, task task.UploadObjectTask)

func (*NullModular) PreCreateBucketApproval 

func (*NullModular) PreCreateBucketApproval(context.Context, task.ApprovalCreateBucketTask) error

func (*NullModular) PreCreateObjectApproval 

func (*NullModular) PreCreateObjectApproval(context.Context, task.ApprovalCreateObjectTask) error

func (*NullModular) PreMigrateBucketApproval added in v0.2.3 

func (*NullModular) PreMigrateBucketApproval(context.Context, task.ApprovalMigrateBucketTask) error

func (*NullModular) PreReplicatePieceApproval 

func (*NullModular) PreReplicatePieceApproval(context.Context, task.ApprovalReplicatePieceTask) error

func (*NullModular) PreResumableUploadObject added in v0.2.3 

func (*NullModular) PreResumableUploadObject(ctx context.Context, task task.ResumableUploadObjectTask) error

func (*NullModular) PreUploadObject 

func (*NullModular) PreUploadObject(ctx context.Context, task task.UploadObjectTask) error

func (*NullModular) QueryBucketMigrate added in v0.2.3 

func (m *NullModular) QueryBucketMigrate(ctx context.Context) (*gfspserver.GfSpQueryBucketMigrateResponse, error)

func (*NullModular) QuerySpExit added in v0.2.3 

func (m *NullModular) QuerySpExit(ctx context.Context) (*gfspserver.GfSpQuerySpExitResponse, error)

func (*NullModular) QueryTask 

func (*NullModular) QueryTask(context.Context, task.TKey) (task.Task, error)

func (*NullModular) QueryTasks 

func (*NullModular) QueryTasks(ctx context.Context, keyPrefix task.TKey) ([]task.Task, error)

func (*NullModular) ReleaseResource 

func (*NullModular) ReleaseResource(context.Context, rcmgr.ResourceScopeSpan)

func (*NullModular) ReserveResource 

func (*NullModular) ReserveResource(context.Context, *rcmgr.ScopeStat) (rcmgr.ResourceScopeSpan, error)

func (*NullModular) Start 

func (*NullModular) Start(context.Context) error

func (*NullModular) Stop 

func (*NullModular) Stop(context.Context) error

func (*NullModular) UpdateUserPublicKey 

func (*NullModular) UpdateUserPublicKey(ctx context.Context, account string, domain string, currentNonce int32, nonce int32, userPublicKey string, expiryDate int64) (bool, error)

func (*NullModular) VerifyAuthentication 

func (*NullModular) VerifyAuthentication(context.Context, AuthOpType, string, string, string) (bool, error)

func (*NullModular) VerifyOffChainSignature 

func (*NullModular) VerifyOffChainSignature(ctx context.Context, account string, domain string, offChainSig string, realMsgToSign string) (bool, error)

type NullReceiveModular 

type NullReceiveModular struct{}

func (*NullReceiveModular) HandleDoneReceivePieceTask 

func (*NullReceiveModular) HandleDoneReceivePieceTask(context.Context, task.ReceivePieceTask) ([]byte, error)

func (*NullReceiveModular) HandleReceivePieceTask 

func (*NullReceiveModular) HandleReceivePieceTask(context.Context, task.ReceivePieceTask, []byte) error

func (*NullReceiveModular) Name 

func (*NullReceiveModular) Name() string

func (*NullReceiveModular) QueryTasks 

func (*NullReceiveModular) QueryTasks(ctx context.Context, keyPrefix task.TKey) ([]task.Task, error)

func (*NullReceiveModular) ReleaseResource 

func (*NullReceiveModular) ReleaseResource(context.Context, rcmgr.ResourceScopeSpan)

func (*NullReceiveModular) ReserveResource 

func (*NullReceiveModular) ReserveResource(context.Context, *rcmgr.ScopeStat) (rcmgr.ResourceScopeSpan, error)

func (*NullReceiveModular) Start 

func (*NullReceiveModular) Start(context.Context) error

func (*NullReceiveModular) Stop 

func (*NullReceiveModular) Stop(context.Context) error

type P2P 

type P2P interface {
	Modular
	// HandleReplicatePieceApproval handles the asking replicate piece approval, it will
	// broadcast the approval to other SPs, waiting the responses. If up to min approved
	// number or max approved number before timeout, it will return the approvals.
	HandleReplicatePieceApproval(ctx context.Context, task task.ApprovalReplicatePieceTask, min, max int32,
		timeout int64) ([]task.ApprovalReplicatePieceTask, error)
	// HandleQueryBootstrap handles the query p2p node bootstrap node info.
	HandleQueryBootstrap(ctx context.Context) ([]string, error)
	// QueryTasks queries replicate piece approval tasks that running on p2p by task sub-key.
	QueryTasks(ctx context.Context, subKey task.TKey) ([]task.Task, error)
}

P2P is an abstract interface to the to do replicate piece approvals between SPs.

type Receiver 

type Receiver interface {
	Modular
	// HandleReceivePieceTask stores piece data into secondary SP.
	HandleReceivePieceTask(ctx context.Context, task task.ReceivePieceTask, data []byte) error
	// HandleDoneReceivePieceTask calculates the secondary bls of the object and sign it, returns to the primary
	// SP for sealed object.
	HandleDoneReceivePieceTask(ctx context.Context, task task.ReceivePieceTask) ([]byte, error)
	// QueryTasks queries replicate piece tasks that running on receiver by task sub-key.
	QueryTasks(ctx context.Context, subKey task.TKey) ([]task.Task, error)
}

Receiver is an abstract interface to receive the piece data from primary SP.

type Signer 

type Signer interface {
	Modular
	// SignCreateBucketApproval signs the MsgCreateBucket for asking create bucket approval.
	SignCreateBucketApproval(ctx context.Context, bucket *storagetypes.MsgCreateBucket) ([]byte, error)
	// SignMigrateBucketApproval signs the MsgMigrateBucket for asking migrate bucket approval
	SignMigrateBucketApproval(ctx context.Context, bucket *storagetypes.MsgMigrateBucket) ([]byte, error)
	// SignCreateObjectApproval signs the MsgCreateObject for asking create object approval.
	SignCreateObjectApproval(ctx context.Context, task *storagetypes.MsgCreateObject) ([]byte, error)
	// SignReplicatePieceApproval signs the ApprovalReplicatePieceTask for asking replicate pieces to secondary SPs.
	SignReplicatePieceApproval(ctx context.Context, task task.ApprovalReplicatePieceTask) ([]byte, error)
	// SignReceivePieceTask signs the ReceivePieceTask for replicating pieces data between SPs.
	SignReceivePieceTask(ctx context.Context, task task.ReceivePieceTask) ([]byte, error)
	// SignSecondarySealBls signs the secondary bls for sealing object.
	SignSecondarySealBls(ctx context.Context, objectID uint64, gvgId uint32, hash [][]byte) ([]byte, error)
	// SignRecoveryPieceTask signs the RecoveryPieceTask for recovering piece data
	SignRecoveryPieceTask(ctx context.Context, task task.RecoveryPieceTask) ([]byte, error)
	// SignP2PPingMsg signs the ping msg for p2p node probing.
	SignP2PPingMsg(ctx context.Context, ping *gfspp2p.GfSpPing) ([]byte, error)
	// SignP2PPongMsg signs the pong msg for p2p to response ping msg.
	SignP2PPongMsg(ctx context.Context, pong *gfspp2p.GfSpPong) ([]byte, error)
	// SealObject signs the MsgSealObject and broadcast the tx to greenfield.
	SealObject(ctx context.Context, object *storagetypes.MsgSealObject) (string, error)
	// RejectUnSealObject signs the MsgRejectSealObject and broadcast the tx to greenfield.
	RejectUnSealObject(ctx context.Context, object *storagetypes.MsgRejectSealObject) (string, error)
	// DiscontinueBucket signs the MsgDiscontinueBucket and broadcast the tx to greenfield.
	DiscontinueBucket(ctx context.Context, bucket *storagetypes.MsgDiscontinueBucket) (string, error)
	// CreateGlobalVirtualGroup signs the MsgCreateGlobalVirtualGroup and broadcast the tx to greenfield.
	CreateGlobalVirtualGroup(ctx context.Context, gvg *virtualgrouptypes.MsgCreateGlobalVirtualGroup) error
	// SignMigratePiece signs the GfSpMigratePieceTask for migrating piece
	SignMigratePiece(ctx context.Context, task *gfsptask.GfSpMigratePieceTask) ([]byte, error)
	// CompleteMigrateBucket signs the MsgCompleteMigrateBucket and broadcast the tx to greenfield.
	CompleteMigrateBucket(ctx context.Context, migrateBucket *storagetypes.MsgCompleteMigrateBucket) (string, error)
	// SignSecondarySPMigrationBucket signs secondary sp bls for bucket migration
	SignSecondarySPMigrationBucket(ctx context.Context, signDoc *storagetypes.SecondarySpMigrationBucketSignDoc) ([]byte, error)
	// SwapOut signs the MsgSwapOut and broadcast the tx to greenfield.
	SwapOut(ctx context.Context, swapOut *virtualgrouptypes.MsgSwapOut) (string, error)
	// SignSwapOut signs the MsgSwapOut for asking swap out approval.
	SignSwapOut(ctx context.Context, swapOut *virtualgrouptypes.MsgSwapOut) ([]byte, error)
	// CompleteSwapOut signs the MsgCompleteSwapOut and broadcast the tx to greenfield.
	CompleteSwapOut(ctx context.Context, completeSwapOut *virtualgrouptypes.MsgCompleteSwapOut) (string, error)
	// SPExit signs the MsgStorageProviderExit and broadcast the tx to greenfield.
	SPExit(ctx context.Context, spExit *virtualgrouptypes.MsgStorageProviderExit) (string, error)
	// CompleteSPExit signs the MsgCompleteStorageProviderExit and broadcast the tx to greenfield.
	CompleteSPExit(ctx context.Context, completeSPExit *virtualgrouptypes.MsgCompleteStorageProviderExit) (string, error)
	// UpdateSPPrice signs the MsgUpdateSpStoragePrice and  broadcast the tx to greenfield.
	UpdateSPPrice(ctx context.Context, price *sptypes.MsgUpdateSpStoragePrice) (string, error)
}

Signer is an abstract interface to handle the signature of SP and on greenfield chain operator. It holds all private keys of one SP. Considering the SP account's sequence number, it must be a singleton.

type TaskExecutor 

type TaskExecutor interface {
	Modular
	// AskTask asks the task by remaining limitation from manager module.
	AskTask(ctx context.Context) error
	// HandleReplicatePieceTask handles ReplicatePieceTask that is asked from manager module.
	HandleReplicatePieceTask(ctx context.Context, task task.ReplicatePieceTask)
	// HandleSealObjectTask handles SealObjectTask that is asked from manager module.
	HandleSealObjectTask(ctx context.Context, task task.SealObjectTask)
	// HandleReceivePieceTask handles the ReceivePieceTask that is asked from manager module.
	// It will confirm the piece data that is synced to secondary SP whether has been sealed.
	HandleReceivePieceTask(ctx context.Context, task task.ReceivePieceTask)
	// HandleGCObjectTask handles the GCObjectTask that is asked from manager module.
	HandleGCObjectTask(ctx context.Context, task task.GCObjectTask)
	// HandleGCZombiePieceTask handles the GCZombiePieceTask that is asked from manager module.
	HandleGCZombiePieceTask(ctx context.Context, task task.GCZombiePieceTask)
	// HandleGCMetaTask handles the GCMetaTask that is asked from manager module.
	HandleGCMetaTask(ctx context.Context, task task.GCMetaTask)
	// HandleMigrateGVGTask handles the MigrateGVGTask that is asked from manager module
	HandleMigrateGVGTask(ctx context.Context, gvgTask task.MigrateGVGTask)
	// ReportTask reports the results or status of running task to manager module.
	ReportTask(ctx context.Context, task task.Task) error
}

TaskExecutor is an abstract interface to handle background tasks. It will ask tasks from manager modular, handle tasks and report the results or status to the manager modular It can handle these tasks: ReplicatePieceTask, SealObjectTask, ReceivePieceTask, GCObjectTask, GCZombiePieceTask, GCMetaTask.

type Uploader 

type Uploader interface {
	Modular
	// PreUploadObject prepares to handle UploadObject, it can do some checks
	// such as checking for duplicates, if limitation of SP has been reached, etc.
	PreUploadObject(ctx context.Context, task task.UploadObjectTask) error
	// HandleUploadObjectTask handles the UploadObject, store payload data into piece store by data stream.
	HandleUploadObjectTask(ctx context.Context, task task.UploadObjectTask, stream io.Reader) error
	// PostUploadObject is called after HandleUploadObjectTask, it can recycle
	// resources, make statistics and do some other operations.
	PostUploadObject(ctx context.Context, task task.UploadObjectTask)

	// PreResumableUploadObject prepares to handle ResumableUploadObject, it can do some checks
	// such as checking for duplicates, if limitation of SP has been reached, etc.
	PreResumableUploadObject(ctx context.Context, task task.ResumableUploadObjectTask) error
	// HandleResumableUploadObjectTask handles the ResumableUploadObject, store payload data into piece store by data stream.
	HandleResumableUploadObjectTask(ctx context.Context, task task.ResumableUploadObjectTask, stream io.Reader) error
	// PostResumableUploadObject is called after HandleResumableUploadObjectTask, it can recycle
	// resources, statistics and other operations.
	PostResumableUploadObject(ctx context.Context, task task.ResumableUploadObjectTask)

	// QueryTasks queries upload object tasks that running on uploading by task sub-key.
	QueryTasks(ctx context.Context, subKey task.TKey) ([]task.Task, error)
}

Uploader is an abstract interface to handle putting object requests from users' account and store their payload data into primary SP piece store.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.