chunker

package
v1.24.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 7, 2025 License: Apache-2.0, MIT Imports: 30 Imported by: 0

Documentation

Index

Constants

View Source
const EntryCacheSize = 20

Entries are 0.5MiB in size, so we do ~10MiB of caching here This cache is only useful in the edge case when entry reads are very slow and time out - this makes retried reads faster

View Source
const NoSkipCacheTTL = 3 * time.Minute

Variables

View Source
var (
	ErrNotFound = errors.New("not found")
)

Functions

func NewEntriesChunkNode

func NewEntriesChunkNode(mhs []multihash.Multihash, next ipld.Link) (datamodel.Node, error)

Types

type InitialChunker

type InitialChunker struct {
	// contains filtered or unexported fields
}

InitialChunker is used for initial entry chain creation. It employs a dual strategy, where it tracks the number of entries, and: For chains with less than longChainThreshold entries, it accumulates entries in memory. which will then form a sorted chain of schema.EntryChunk nodes. This allows creation of ad chains which are read from the index database. For chains with more than longChainThreshold entries, it creates a chain of schema.EntryChunk nodes in car order, noting where each chunk starts and ends in the car file.

func NewInitialChunker

func NewInitialChunker() *InitialChunker

func (*InitialChunker) Accept

func (c *InitialChunker) Accept(mh multihash.Multihash, startOff int64, entryLen uint64) error

func (*InitialChunker) Finish

func (c *InitialChunker) Finish(ctx context.Context, db *harmonydb.DB, pieceCid cid.Cid) (ipld.Link, error)

type ServeChunker

type ServeChunker struct {
	// contains filtered or unexported fields
}

func NewServeChunker

func NewServeChunker(db *harmonydb.DB, pieceProvider *pieceprovider.PieceProvider, indexStore *indexstore.IndexStore, cpr *cachedreader.CachedPieceReader) *ServeChunker

func (*ServeChunker) GetEntry

func (p *ServeChunker) GetEntry(ctx context.Context, block cid.Cid) (b []byte, err error)

GetEntry retrieves an entry from the provider's database based on the given block CID and provider ID. It returns the entry data as a byte slice, or an error if the entry is not found or an error occurs during retrieval. If the entry is stored as a CAR file, it reconstructs the chunk from the CAR file.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL