chunk

package
v0.5.9-0...-ba7202b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 11, 2022 License: GPL-3.0 Imports: 15 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DefaultSize   = 4096
	MaxPO         = 16
	AddressLength = 32
)

Variables

View Source 
var (
	ErrChunkNotFound = errors.New("chunk not found")
	ErrChunkInvalid  = errors.New("invalid chunk")
)
View Source 
var (
	TagUidFunc     = rand.Uint32
	TagNotFoundErr = errors.New("tag not found")
)
View Source 
var ZeroAddr = Address(common.Hash{}.Bytes())

Functions

func Proximity 

func Proximity(one, other []byte) (ret int)

Proximity returns the proximity order of the MSB distance between x and y

The distance metric MSB(x, y) of two equal length byte sequences x an y is the value of the binary integer cast of the x^y, ie., x and y bitwise xor-ed. the binary cast is big endian: most significant bit first (=MSB).

Proximity(x, y) is a discrete logarithmic scaling of the MSB distance. It is defined as the reverse rank of the integer part of the base 2 logarithm of the distance. It is calculated by counting the number of common leading zeros in the (MSB) binary representation of the x^y.

(0 farthest, 255 closest, 256 self)

Types

type Address 

type Address []byte

func (Address) Hex 

func (a Address) Hex() string

func (Address) Log 

func (a Address) Log() string

func (Address) MarshalJSON 

func (a Address) MarshalJSON() (out []byte, err error)

func (Address) String 

func (a Address) String() string

func (*Address) UnmarshalJSON 

func (a *Address) UnmarshalJSON(value []byte) error

type Chunk 

type Chunk interface {
	Address() Address
	Data() []byte
	PinCounter() uint64
	WithPinCounter(p uint64) Chunk
	TagID() uint32
	WithTagID(t uint32) Chunk
}

func NewChunk 

func NewChunk(addr Address, data []byte) Chunk

type Descriptor 

type Descriptor struct {
	Address Address
	BinID   uint64
}

Descriptor holds information required for Pull syncing. This struct is provided by subscribing to pull index.

func (*Descriptor) String 

func (d *Descriptor) String() string

type ModeGet 

type ModeGet int

ModeGet enumerates different Getter modes. 

const (
	// ModeGetRequest: when accessed for retrieval
	ModeGetRequest ModeGet = iota
	// ModeGetSync: when accessed for syncing or proof of custody request
	ModeGetSync
	// ModeGetLookup: when accessed to lookup a a chunk in feeds or other places
	ModeGetLookup
	// ModeGetPin: used when a pinned chunk is accessed
	ModeGetPin
)

Getter modes.

func (ModeGet) String 

func (m ModeGet) String() string

type ModePut 

type ModePut int

ModePut enumerates different Putter modes. 

const (
	// ModePutRequest: when a chunk is received as a result of retrieve request and delivery
	ModePutRequest ModePut = iota
	// ModePutSync: when a chunk is received via syncing
	ModePutSync
	// ModePutUpload: when a chunk is created by local upload
	ModePutUpload
)

Putter modes.

func (ModePut) String 

func (m ModePut) String() string

type ModeSet 

type ModeSet int

ModeSet enumerates different Setter modes. 

const (
	// ModeSetAccess: when an update request is received for a chunk or chunk is retrieved for delivery
	ModeSetAccess ModeSet = iota
	// ModeSetSyncPush: when a push sync receipt is received for a chunk
	ModeSetSyncPush
	// ModeSetSyncPull: when a chunk is added to a pull sync batch
	ModeSetSyncPull
	// ModeSetRemove: when a chunk is removed
	ModeSetRemove
	// ModeSetPin: when a chunk is pinned during upload or separately
	ModeSetPin
	// ModeSetUnpin: when a chunk is unpinned using a command locally
	ModeSetUnpin
)

Setter modes.

func (ModeSet) String 

func (m ModeSet) String() string

type State 

type State = uint32

State is the enum type for chunk states 

const (
	StateSplit  State = iota // chunk has been processed by filehasher/swarm safe call
	StateStored              // chunk stored locally
	StateSeen                // chunk previously seen
	StateSent                // chunk sent to neighbourhood
	StateSynced              // proof is received; chunk removed from sync db; chunk is available everywhere
)

type Store 

type Store interface {
	Get(ctx context.Context, mode ModeGet, addr Address) (ch Chunk, err error)
	GetMulti(ctx context.Context, mode ModeGet, addrs ...Address) (ch []Chunk, err error)
	Put(ctx context.Context, mode ModePut, chs ...Chunk) (exist []bool, err error)
	Has(ctx context.Context, addr Address) (yes bool, err error)
	HasMulti(ctx context.Context, addrs ...Address) (yes []bool, err error)
	Set(ctx context.Context, mode ModeSet, addrs ...Address) (err error)
	LastPullSubscriptionBinID(bin uint8) (id uint64, err error)
	SubscribePull(ctx context.Context, bin uint8, since, until uint64) (c <-chan Descriptor, stop func())
	Close() (err error)
}

type Tag 

type Tag struct {
	Total  int64 // total chunks belonging to a tag
	Split  int64 // number of chunks already processed by splitter for hashing
	Seen   int64 // number of chunks already seen
	Stored int64 // number of chunks already stored locally
	Sent   int64 // number of chunks sent for push syncing
	Synced int64 // number of chunks synced with proof

	Uid       uint32    // a unique identifier for this tag
	Anonymous bool      // indicates if the tag is anonymous (i.e. if only pull sync should be used)
	Name      string    // a name tag for this tag
	Address   Address   // the associated swarm hash for this tag
	StartedAt time.Time // tag started to calculate ETA
	// contains filtered or unexported fields
}

Tag represents info on the status of new chunks

func NewTag 

func NewTag(uid uint32, s string, total int64, anon bool) *Tag

NewTag creates a new tag, and returns it

func (*Tag) Context 

func (t *Tag) Context() context.Context

Context accessor

func (*Tag) Done 

func (t *Tag) Done(s State) bool

Done returns true if tag is complete wrt the state given as argument

func (*Tag) DoneSplit 

func (t *Tag) DoneSplit(address Address) int64

DoneSplit sets total count to SPLIT count and sets the associated swarm hash for this tag is meant to be called when splitter finishes for input streams of unknown size

func (*Tag) ETA 

func (t *Tag) ETA(state State) (time.Time, error)

ETA returns the time of completion estimated based on time passed and rate of completion

func (*Tag) FinishRootSpan 

func (t *Tag) FinishRootSpan()

FinishRootSpan closes the pushsync span of the tags

func (*Tag) Get 

func (t *Tag) Get(state State) int64

Get returns the count for a state on a tag

func (*Tag) Inc 

func (t *Tag) Inc(state State)

Inc increments the count for a state

func (*Tag) IncN 

func (t *Tag) IncN(state State, n int)

IncN increments the count for a state

func (*Tag) MarshalBinary 

func (tag *Tag) MarshalBinary() (data []byte, err error)

MarshalBinary marshals the tag into a byte slice

func (*Tag) Status 

func (t *Tag) Status(state State) (int64, int64, error)

Status returns the value of state and the total count

func (*Tag) TotalCounter 

func (t *Tag) TotalCounter() int64

GetTotal returns the total count

func (*Tag) UnmarshalBinary 

func (tag *Tag) UnmarshalBinary(buffer []byte) error

UnmarshalBinary unmarshals a byte slice into a tag

func (*Tag) WaitTillDone 

func (t *Tag) WaitTillDone(ctx context.Context, s State) error

WaitTillDone returns without error once the tag is complete wrt the state given as argument it returns an error if the context is done

type Tags 

type Tags struct {
	// contains filtered or unexported fields
}

Tags hold tag information indexed by a unique random uint32

func NewTags 

func NewTags() *Tags

NewTags creates a tags object

func (*Tags) All 

func (ts *Tags) All() (t []*Tag)

All returns all existing tags in Tags' sync.Map Note that tags are returned in no particular order

func (*Tags) Create 

func (ts *Tags) Create(s string, total int64, anon bool) (*Tag, error)

Create creates a new tag, stores it by the name and returns it it returns an error if the tag with this name already exists

func (*Tags) Delete 

func (ts *Tags) Delete(k interface{})

func (*Tags) Get 

func (ts *Tags) Get(uid uint32) (*Tag, error)

Get returns the underlying tag for the uid or an error if not found

func (*Tags) GetByAddress 

func (ts *Tags) GetByAddress(address Address) (*Tag, error)

GetByAddress returns the latest underlying tag for the address or an error if not found

func (*Tags) GetFromContext 

func (ts *Tags) GetFromContext(ctx context.Context) (*Tag, error)

GetFromContext gets a tag from the tag uid stored in the context

func (*Tags) MarshalJSON 

func (ts *Tags) MarshalJSON() (out []byte, err error)

func (*Tags) Range 

func (ts *Tags) Range(fn func(k, v interface{}) bool)

Range exposes sync.Map's iterator

func (*Tags) UnmarshalJSON 

func (ts *Tags) UnmarshalJSON(value []byte) error

type Validator 

type Validator interface {
	Validate(ch Chunk) bool
}

Validator validates a chunk.

type ValidatorStore 

type ValidatorStore struct {
	Store
	// contains filtered or unexported fields
}

ValidatorStore encapsulates Store by decorating the Put method with validators check.

func NewValidatorStore 

func NewValidatorStore(store Store, validators ...Validator) (s *ValidatorStore)

NewValidatorStore returns a new ValidatorStore which uses provided validators to validate chunks on Put.

func (*ValidatorStore) Put 

func (s *ValidatorStore) Put(ctx context.Context, mode ModePut, chs ...Chunk) (exist []bool, err error)

Put overrides Store put method with validators check. For Put to succeed, all provided chunks must be validated with true by one of the validators.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.