downloader

package
v1.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 20, 2020 License: Apache-2.0 Imports: 30 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type ClientStreamWriter added in v1.0.1 

type ClientStreamWriter struct {
	// contains filtered or unexported fields
}

ClientWriter writes a file for uploading and a target file.

func NewClientStreamWriter added in v1.0.1 

func NewClientStreamWriter(clientQueue, notifyQueue queue.Queue, api api.SupernodeAPI, cfg *config.Config) *ClientStreamWriter

NewClientStreamWriter creates and initialize a ClientStreamWriter instance.

func (*ClientStreamWriter) PostRun added in v1.0.1 

func (csw *ClientStreamWriter) PostRun(ctx context.Context) (err error)

func (*ClientStreamWriter) PreRun added in v1.0.1 

func (csw *ClientStreamWriter) PreRun(ctx context.Context) (err error)

func (*ClientStreamWriter) Read added in v1.0.1 

func (csw *ClientStreamWriter) Read(p []byte) (n int, err error)

func (*ClientStreamWriter) Run added in v1.0.1 

func (csw *ClientStreamWriter) Run(ctx context.Context)

Run starts writing pipe.

func (*ClientStreamWriter) Wait added in v1.0.1 

func (csw *ClientStreamWriter) Wait()

Wait for Run whether is finished.

type ClientWriter 

type ClientWriter struct {
	// contains filtered or unexported fields
}

ClientWriter writes a file for uploading and a target file.

func (*ClientWriter) PostRun added in v1.0.1 

func (cw *ClientWriter) PostRun(ctx context.Context) (err error)

func (*ClientWriter) PreRun added in v1.0.1 

func (cw *ClientWriter) PreRun(ctx context.Context) (err error)

func (*ClientWriter) Run 

func (cw *ClientWriter) Run(ctx context.Context)

Run starts writing downloading file.

func (*ClientWriter) Wait 

func (cw *ClientWriter) Wait()

Wait for Run whether is finished.

type P2PDownloader 

type P2PDownloader struct {
	// API holds the instance of SupernodeAPI to interact with supernode.
	API api.SupernodeAPI
	// Register holds the instance of SupernodeRegister.
	Register regist.SupernodeRegister
	// RegisterResult indicates the result set of registering to supernode.
	RegisterResult *regist.RegisterResult
	// contains filtered or unexported fields
}

P2PDownloader is one implementation of Downloader that uses p2p pattern to download files.

func NewP2PDownloader 

func NewP2PDownloader(cfg *config.Config,
	api api.SupernodeAPI,
	register regist.SupernodeRegister,
	result *regist.RegisterResult) *P2PDownloader

NewP2PDownloader creates a P2PDownloader.

func (*P2PDownloader) Cleanup 

func (p2p *P2PDownloader) Cleanup()

Cleanup cleans all temporary resources generated by executing Run.

func (*P2PDownloader) GetNode 

func (p2p *P2PDownloader) GetNode() string

GetNode returns supernode ip.

func (*P2PDownloader) GetTaskID 

func (p2p *P2PDownloader) GetTaskID() string

GetTaskID returns downloading taskID.

func (*P2PDownloader) Run 

func (p2p *P2PDownloader) Run(ctx context.Context) error

Run starts to download the file.

func (*P2PDownloader) RunStream added in v1.0.1 

func (p2p *P2PDownloader) RunStream(ctx context.Context) (io.Reader, error)

RunStream starts to download the file, but return a io.Reader instead of writing a file to local disk.

type Piece 

type Piece struct {
	// TaskID a string which represents a unique task.
	TaskID string `json:"taskID"`

	// SuperNode indicates the IP address of the currently registered supernode.
	SuperNode string `json:"superNode"`

	// DstCid means the clientID of the target peer node for downloading the piece.
	DstCid string `json:"dstCid"`

	// Range indicates the range of specific piece in the task, example "0-45565".
	Range string `json:"range"`

	// Result of the piece downloaded.
	Result int `json:"result"`

	// Status of the downloading task.
	Status int `json:"status"`

	// PieceSize the length of the piece.
	PieceSize int32 `json:"pieceSize"`

	// PieceNum represents the position of the piece in the pieces list by cutting files.
	PieceNum int `json:"pieceNum"`

	// Content uses a buffer to temporarily store the piece content.
	Content *pool.Buffer `json:"-"`
	// contains filtered or unexported fields
}

Piece contains all information of a piece.

func NewPiece 

func NewPiece(taskID, node, dstCid, pieceRange string, result, status int, cdnSource apiTypes.CdnSource) *Piece

NewPiece creates a Piece.

func NewPieceContent 

func NewPieceContent(taskID, node, dstCid, pieceRange string,
	result, status int, contents *pool.Buffer, cdnSource apiTypes.CdnSource) *Piece

NewPieceContent creates a Piece with specified content.

func NewPieceSimple 

func NewPieceSimple(taskID string, node string, status int, cdnSource apiTypes.CdnSource) *Piece

NewPieceSimple creates a Piece with default value.

func (*Piece) ContentLength added in v1.0.3 

func (p *Piece) ContentLength() int64

ContentLength returns the content length.

func (*Piece) IncWriter added in v1.0.4 

func (p *Piece) IncWriter()

IncWriter increase a writer for the piece.

func (*Piece) RawContent 

func (p *Piece) RawContent(noWrapper bool) *bytes.Buffer

RawContent returns raw contents, If the piece has wrapper, and the piece content will remove the head and tail.

func (*Piece) String 

func (p *Piece) String() string

func (*Piece) TryResetContent added in v1.0.4 

func (p *Piece) TryResetContent()

ResetContent reset contents and returns it back to buffer pool.

func (*Piece) WriteTo added in v1.0.4 

func (p *Piece) WriteTo(w io.Writer, noWrapper bool) (n int64, err error)

WriteTo writes piece raw data in content buffer to w. If the piece has wrapper, the piece content will remove the head and tail before writing.

type PieceWriter added in v1.0.1 

type PieceWriter interface {
	// PreRun initializes PieceWriter
	PreRun(ctx context.Context) error

	// Run starts to process piece data in background
	Run(ctx context.Context)

	// PostRun will run when finish a task
	PostRun(ctx context.Context) error

	// Wait will block util all piece are processed
	Wait()
}

PieceWriter will be used in p2p downloader we provide 2 implementations, one for downloading file, one for streaming

func NewClientWriter 

func NewClientWriter(clientFilePath, serviceFilePath string,
	clientQueue, notifyQueue queue.Queue,
	api api.SupernodeAPI, cfg *config.Config, cdnSource apiTypes.CdnSource) PieceWriter

NewClientWriter creates and initialize a ClientWriter instance.

type PowerClient 

type PowerClient struct {
	// contains filtered or unexported fields
}

PowerClient downloads file from dragonfly.

func (*PowerClient) ClientError added in v0.3.1 

func (pc *PowerClient) ClientError() *types.ClientErrorRequest

ClientError returns the client error if occurred

func (*PowerClient) Run 

func (pc *PowerClient) Run() error

Run starts run the task.

type TargetWriter 

type TargetWriter struct {
	// contains filtered or unexported fields
}

TargetWriter writes downloading file to disk.

func NewTargetWriter 

func NewTargetWriter(dst string, q queue.Queue, cfg *config.Config, cdnSource apiTypes.CdnSource) (*TargetWriter, error)

NewTargetWriter creates and initialize a TargetWriter instance.

func (*TargetWriter) Run 

func (tw *TargetWriter) Run(ctx context.Context)

Run starts writing downloading file.

func (*TargetWriter) Wait 

func (tw *TargetWriter) Wait()

Wait the Run is finished.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.