netmap

package
v1.22.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 15, 2021 License: Apache-2.0 Imports: 6 Imported by: 19

Documentation

Overview

Package netmap provides routines for working with netmap and placement policy. Work is done in 4 steps: 1. Create context containing results shared between steps. 2. Processing filters. 3. Processing selectors. 4. Processing replicas.

Each step depends only on previous ones.

Index

Constants

View Source 
const (
	CapacityAttr = "Capacity"
	PriceAttr    = "Price"
)

Enumeration of well-known attributes.

View Source 
const MainFilterName = "*"

MainFilterName is a name of the filter which points to the whole netmap.

Variables

View Source 
var (
	ErrMissingField      = errors.New("netmap: nil field")
	ErrInvalidFilterName = errors.New("netmap: filter name is invalid")
	ErrInvalidNumber     = errors.New("netmap: number value expected")
	ErrInvalidFilterOp   = errors.New("netmap: invalid filter operation")
	ErrFilterNotFound    = errors.New("netmap: filter not found")
	ErrNonEmptyFilters   = errors.New("netmap: simple filter must no contain sub-filters")
	ErrNotEnoughNodes    = errors.New("netmap: not enough nodes to SELECT from")
	ErrSelectorNotFound  = errors.New("netmap: selector not found")
	ErrUnnamedTopFilter  = errors.New("netmap: all filters on top level must be named")
)

Various validation errors.

Functions

func GetBucketWeight 

func GetBucketWeight(ns Nodes, a aggregator, wf weightFunc) float64

GetBucketWeight computes weight for a Bucket.

func GetDefaultWeightFunc 

func GetDefaultWeightFunc(ns Nodes) weightFunc

GetDefaultWeightFunc returns default weighting function.

func GetNodesCount 

func GetNodesCount(_ *PlacementPolicy, s *Selector) (int, int)

GetNodesCount returns amount of buckets and minimum number of nodes in every bucket for the given selector.

Types

type Clause 

type Clause uint32

Clause is an enumeration of selector modifiers that shows how the node set will be formed. 

const (
	ClauseUnspecified Clause = iota

	// ClauseSame is a selector modifier to select only nodes having the same value of bucket attribute.
	ClauseSame

	// ClauseDistinct is a selector modifier to select nodes having different values of bucket attribute.
	ClauseDistinct
)

func ClauseFromV2 

func ClauseFromV2(c netmap.Clause) Clause

ClauseFromV2 converts v2 Clause to Clause.

func (Clause) String 

func (c Clause) String() string

func (Clause) ToV2 

func (c Clause) ToV2() netmap.Clause

ToV2 converts Clause to v2 Clause.

type ContainerNodes 

type ContainerNodes interface {
	Replicas() []Nodes
	Flatten() Nodes
}

ContainerNodes represents nodes in the container.

type Context 

type Context struct {
	// Netmap is a netmap structure to operate on.
	Netmap *Netmap
	// Filters stores processed filters.
	Filters map[string]*Filter
	// Selectors stores processed selectors.
	Selectors map[string]*Selector
	// Selections stores result of selector processing.
	Selections map[string][]Nodes
	// contains filtered or unexported fields
}

Context contains references to named filters and cached numeric values.

func NewContext 

func NewContext(nm *Netmap) *Context

NewContext creates new context. It contains various caches. In future it may create hierarchical netmap structure to work with.

type Filter 

type Filter netmap.Filter

Filter represents v2-compatible netmap filter.

func NewFilter 

func NewFilter() *Filter

NewFilter creates and returns new Filter instance.

func NewFilterFromV2 

func NewFilterFromV2(f *netmap.Filter) *Filter

NewFilterFromV2 converts v2 Filter to Filter.

func (*Filter) InnerFilters 

func (f *Filter) InnerFilters() []*Filter

InnerFilters returns list of inner filters.

func (*Filter) Key 

func (f *Filter) Key() string

Key returns key to filter.

func (*Filter) Marshal 

func (f *Filter) Marshal(b ...[]byte) ([]byte, error)

Marshal marshals Filter into a protobuf binary form.

Buffer is allocated when the argument is empty. Otherwise, the first buffer is used.

func (*Filter) MarshalJSON 

func (f *Filter) MarshalJSON() ([]byte, error)

MarshalJSON encodes Filter to protobuf JSON format.

func (*Filter) Name 

func (f *Filter) Name() string

Name returns filter name.

func (*Filter) Operation 

func (f *Filter) Operation() Operation

Operation returns filtering operation.

func (*Filter) SetInnerFilters 

func (f *Filter) SetInnerFilters(fs ...*Filter)

SetInnerFilters sets list of inner filters.

func (*Filter) SetKey 

func (f *Filter) SetKey(key string)

SetKey sets key to filter.

func (*Filter) SetName 

func (f *Filter) SetName(name string)

SetName sets filter name.

func (*Filter) SetOperation 

func (f *Filter) SetOperation(op Operation)

SetOperation sets filtering operation.

func (*Filter) SetValue 

func (f *Filter) SetValue(val string)

SetValue sets value to match.

func (*Filter) ToV2 

func (f *Filter) ToV2() *netmap.Filter

ToV2 converts Filter to v2 Filter.

func (*Filter) Unmarshal 

func (f *Filter) Unmarshal(data []byte) error

Unmarshal unmarshals protobuf binary representation of Filter.

func (*Filter) UnmarshalJSON 

func (f *Filter) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes Filter from protobuf JSON format.

func (*Filter) Value 

func (f *Filter) Value() string

Value returns value to match.

type Netmap 

type Netmap struct {
	Nodes Nodes
}

Netmap represents netmap which contains preprocessed nodes.

func NewNetmap 

func NewNetmap(nodes Nodes) (*Netmap, error)

NewNetmap constructs netmap from the list of raw nodes.

func (*Netmap) GetContainerNodes 

func (m *Netmap) GetContainerNodes(p *PlacementPolicy, pivot []byte) (ContainerNodes, error)

GetContainerNodes returns nodes corresponding to each replica. Order of returned nodes corresponds to order of replicas in p. pivot is a seed for HRW sorting.

func (*Netmap) GetPlacementVectors 

func (m *Netmap) GetPlacementVectors(cnt ContainerNodes, pivot []byte) ([]Nodes, error)

GetPlacementVectors returns placement vectors for an object given containerNodes cnt.

type Node 

type Node struct {
	ID       uint64
	Index    int
	Capacity uint64
	Price    uint64
	AttrMap  map[string]string

	*NodeInfo
}

Node is a wrapper over NodeInfo.

func (*Node) Attribute 

func (n *Node) Attribute(k string) string

Attribute returns value of attribute k.

func (Node) Hash 

func (n Node) Hash() uint64

Hash is a function from hrw.Hasher interface. It is implemented to support weighted hrw therefore sort function sorts nodes based on their `N` value.

type NodeAttribute 

type NodeAttribute netmap.Attribute

NodeAttribute represents v2 compatible attribute of the NeoFS Storage Node.

func NewNodeAttribute 

func NewNodeAttribute() *NodeAttribute

NewNodeAttribute creates and returns new NodeAttribute instance.

func NewNodeAttributeFromV2 

func NewNodeAttributeFromV2(a *netmap.Attribute) *NodeAttribute

NodeAttributeFromV2 converts v2 node Attribute to NodeAttribute.

func (*NodeAttribute) Key 

func (a *NodeAttribute) Key() string

Key returns key to the node attribute.

func (*NodeAttribute) Marshal 

func (a *NodeAttribute) Marshal(b ...[]byte) ([]byte, error)

Marshal marshals NodeAttribute into a protobuf binary form.

Buffer is allocated when the argument is empty. Otherwise, the first buffer is used.

func (*NodeAttribute) MarshalJSON 

func (a *NodeAttribute) MarshalJSON() ([]byte, error)

MarshalJSON encodes NodeAttribute to protobuf JSON format.

func (*NodeAttribute) ParentKeys 

func (a *NodeAttribute) ParentKeys() []string

ParentKeys returns list of parent keys.

func (*NodeAttribute) SetKey 

func (a *NodeAttribute) SetKey(key string)

SetKey sets key to the node attribute.

func (*NodeAttribute) SetParentKeys 

func (a *NodeAttribute) SetParentKeys(keys ...string)

SetParentKeys sets list of parent keys.

func (*NodeAttribute) SetValue 

func (a *NodeAttribute) SetValue(val string)

SetValue sets value of the node attribute.

func (*NodeAttribute) ToV2 

func (a *NodeAttribute) ToV2() *netmap.Attribute

ToV2 converts NodeAttribute to v2 node Attribute.

func (*NodeAttribute) Unmarshal 

func (a *NodeAttribute) Unmarshal(data []byte) error

Unmarshal unmarshals protobuf binary representation of NodeAttribute.

func (*NodeAttribute) UnmarshalJSON 

func (a *NodeAttribute) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes NodeAttribute from protobuf JSON format.

func (*NodeAttribute) Value 

func (a *NodeAttribute) Value() string

Value returns value of the node attribute.

type NodeInfo 

type NodeInfo netmap.NodeInfo

NodeInfo represents v2 compatible descriptor of the NeoFS node.

func NewNodeInfo 

func NewNodeInfo() *NodeInfo

NewNodeInfo creates and returns new NodeInfo instance.

func NewNodeInfoFromV2 

func NewNodeInfoFromV2(i *netmap.NodeInfo) *NodeInfo

NewNodeInfoFromV2 converts v2 NodeInfo to NodeInfo.

func (*NodeInfo) Address 

func (i *NodeInfo) Address() string

Address returns network endpoint address of the node.

func (*NodeInfo) Attributes 

func (i *NodeInfo) Attributes() []*NodeAttribute

Attributes returns list of the node attributes.

func (*NodeInfo) Marshal 

func (i *NodeInfo) Marshal(b ...[]byte) ([]byte, error)

Marshal marshals NodeInfo into a protobuf binary form.

Buffer is allocated when the argument is empty. Otherwise, the first buffer is used.

func (*NodeInfo) MarshalJSON 

func (i *NodeInfo) MarshalJSON() ([]byte, error)

MarshalJSON encodes NodeInfo to protobuf JSON format.

func (*NodeInfo) PublicKey 

func (i *NodeInfo) PublicKey() []byte

PublicKey returns public key of the node in a binary format.

func (*NodeInfo) SetAddress 

func (i *NodeInfo) SetAddress(addr string)

SetAddress sets network endpoint address of the node.

func (*NodeInfo) SetAttributes 

func (i *NodeInfo) SetAttributes(as ...*NodeAttribute)

SetAttributes sets list of the node attributes.

func (*NodeInfo) SetPublicKey 

func (i *NodeInfo) SetPublicKey(key []byte)

SetPublicKey sets public key of the node in a binary format.

func (*NodeInfo) SetState 

func (i *NodeInfo) SetState(s NodeState)

SetState sets node state.

func (*NodeInfo) State 

func (i *NodeInfo) State() NodeState

State returns node state.

func (*NodeInfo) ToV2 

func (i *NodeInfo) ToV2() *netmap.NodeInfo

ToV2 converts NodeInfo to v2 NodeInfo.

func (*NodeInfo) Unmarshal 

func (i *NodeInfo) Unmarshal(data []byte) error

Unmarshal unmarshals protobuf binary representation of NodeInfo.

func (*NodeInfo) UnmarshalJSON 

func (i *NodeInfo) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes NodeInfo from protobuf JSON format.

type NodeState 

type NodeState uint32

NodeState is an enumeration of various states of the NeoFS node. 

const (

	// NodeStateOffline is network unavailable state.
	NodeStateOffline NodeState

	// NodeStateOnline is an active state in the network.
	NodeStateOnline
)

func NodeStateFromV2 

func NodeStateFromV2(s netmap.NodeState) NodeState

NodeStateFromV2 converts v2 NodeState to NodeState.

func (NodeState) String 

func (s NodeState) String() string

func (NodeState) ToV2 

func (s NodeState) ToV2() netmap.NodeState

ToV2 converts NodeState to v2 NodeState.

type Nodes 

type Nodes []*Node

Nodes represents slice of graph leafs.

func NodesFromInfo 

func NodesFromInfo(infos []NodeInfo) Nodes

NodesFromInfo converts slice of NodeInfo to a generic node slice.

func (Nodes) Weights 

func (n Nodes) Weights(wf weightFunc) []float64

Weights returns slice of nodes weights W.

type Operation 

type Operation uint32

Operation is an enumeration of v2-compatible filtering operations. 

const (

	// OpEQ is an "Equal" operation.
	OpEQ Operation

	// OpNE is a "Not equal" operation.
	OpNE

	// OpGT is a "Greater than" operation.
	OpGT

	// OpGE is a "Greater than or equal to" operation.
	OpGE

	// OpLT is a "Less than" operation.
	OpLT

	// OpLE is a "Less than or equal to" operation.
	OpLE

	// OpOR is an "OR" operation.
	OpOR

	// OpAND is an "AND" operation.
	OpAND
)

func OperationFromV2 

func OperationFromV2(op netmap.Operation) Operation

OperationFromV2 converts v2 Operation to Operation.

func (Operation) String 

func (op Operation) String() string

func (Operation) ToV2 

func (op Operation) ToV2() netmap.Operation

ToV2 converts Operation to v2 Operation.

type PlacementPolicy 

type PlacementPolicy netmap.PlacementPolicy

PlacementPolicy represents v2-compatible placement policy.

func NewPlacementPolicy 

func NewPlacementPolicy() *PlacementPolicy

NewPlacementPolicy creates and returns new PlacementPolicy instance.

func NewPlacementPolicyFromV2 

func NewPlacementPolicyFromV2(f *netmap.PlacementPolicy) *PlacementPolicy

NewPlacementPolicyFromV2 converts v2 PlacementPolicy to PlacementPolicy.

func (*PlacementPolicy) ContainerBackupFactor 

func (p *PlacementPolicy) ContainerBackupFactor() uint32

ContainerBackupFactor returns container backup factor.

func (*PlacementPolicy) Filters 

func (p *PlacementPolicy) Filters() []*Filter

Filters returns list of named filters to reference in selectors.

func (*PlacementPolicy) Marshal 

func (p *PlacementPolicy) Marshal(b ...[]byte) ([]byte, error)

Marshal marshals PlacementPolicy into a protobuf binary form.

Buffer is allocated when the argument is empty. Otherwise, the first buffer is used.

func (*PlacementPolicy) MarshalJSON 

func (p *PlacementPolicy) MarshalJSON() ([]byte, error)

MarshalJSON encodes PlacementPolicy to protobuf JSON format.

func (*PlacementPolicy) Replicas 

func (p *PlacementPolicy) Replicas() []*Replica

Replicas returns list of object replica descriptors.

func (*PlacementPolicy) Selectors 

func (p *PlacementPolicy) Selectors() []*Selector

Selector returns set of selectors to form the container's nodes subset.

func (*PlacementPolicy) SetContainerBackupFactor 

func (p *PlacementPolicy) SetContainerBackupFactor(f uint32)

SetContainerBackupFactor sets container backup factor.

func (*PlacementPolicy) SetFilters 

func (p *PlacementPolicy) SetFilters(fs ...*Filter)

SetFilters sets list of named filters to reference in selectors.

func (*PlacementPolicy) SetReplicas 

func (p *PlacementPolicy) SetReplicas(rs ...*Replica)

SetReplicas sets list of object replica descriptors.

func (*PlacementPolicy) SetSelectors 

func (p *PlacementPolicy) SetSelectors(ss ...*Selector)

SetSelectors sets set of selectors to form the container's nodes subset.

func (*PlacementPolicy) ToV2 

func (p *PlacementPolicy) ToV2() *netmap.PlacementPolicy

ToV2 converts PlacementPolicy to v2 PlacementPolicy.

func (*PlacementPolicy) Unmarshal 

func (p *PlacementPolicy) Unmarshal(data []byte) error

Unmarshal unmarshals protobuf binary representation of PlacementPolicy.

func (*PlacementPolicy) UnmarshalJSON 

func (p *PlacementPolicy) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes PlacementPolicy from protobuf JSON format.

type Replica 

type Replica netmap.Replica

Replica represents v2-compatible object replica descriptor.

func NewReplica 

func NewReplica() *Replica

NewReplica creates and returns new Replica instance.

func NewReplicaFromV2 

func NewReplicaFromV2(f *netmap.Replica) *Replica

NewReplicaFromV2 converts v2 Replica to Replica.

func (*Replica) Count 

func (r *Replica) Count() uint32

Count returns number of object replicas.

func (*Replica) Marshal 

func (r *Replica) Marshal(b ...[]byte) ([]byte, error)

Marshal marshals Replica into a protobuf binary form.

Buffer is allocated when the argument is empty. Otherwise, the first buffer is used.

func (*Replica) MarshalJSON 

func (r *Replica) MarshalJSON() ([]byte, error)

MarshalJSON encodes Replica to protobuf JSON format.

func (*Replica) Selector 

func (r *Replica) Selector() string

Selector returns name of selector bucket to put replicas.

func (*Replica) SetCount 

func (r *Replica) SetCount(c uint32)

SetCount sets number of object replicas.

func (*Replica) SetSelector 

func (r *Replica) SetSelector(s string)

SetSelector sets name of selector bucket to put replicas.

func (*Replica) ToV2 

func (r *Replica) ToV2() *netmap.Replica

ToV2 converts Replica to v2 Replica.

func (*Replica) Unmarshal 

func (r *Replica) Unmarshal(data []byte) error

Unmarshal unmarshals protobuf binary representation of Replica.

func (*Replica) UnmarshalJSON 

func (r *Replica) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes Replica from protobuf JSON format.

type Selector 

type Selector netmap.Selector

Selector represents v2-compatible netmap selector.

func NewSelector 

func NewSelector() *Selector

NewSelector creates and returns new Selector instance.

func NewSelectorFromV2 

func NewSelectorFromV2(f *netmap.Selector) *Selector

NewSelectorFromV2 converts v2 Selector to Selector.

func (*Selector) Attribute 

func (s *Selector) Attribute() string

Attribute returns attribute bucket to select from.

func (*Selector) Clause 

func (s *Selector) Clause() Clause

Clause returns modifier showing how to form a bucket.

func (*Selector) Count 

func (s *Selector) Count() uint32

Count returns count of nodes to select from bucket.

func (*Selector) Filter 

func (s *Selector) Filter() string

Filter returns filter reference to select from.

func (*Selector) Marshal 

func (s *Selector) Marshal(b ...[]byte) ([]byte, error)

Marshal marshals Selector into a protobuf binary form.

Buffer is allocated when the argument is empty. Otherwise, the first buffer is used.

func (*Selector) MarshalJSON 

func (s *Selector) MarshalJSON() ([]byte, error)

MarshalJSON encodes Selector to protobuf JSON format.

func (*Selector) Name 

func (s *Selector) Name() string

Name returns selector name.

func (*Selector) SetAttribute 

func (s *Selector) SetAttribute(a string)

SetAttribute sets attribute bucket to select from.

func (*Selector) SetClause 

func (s *Selector) SetClause(c Clause)

SetClause sets modifier showing how to form a bucket.

func (*Selector) SetCount 

func (s *Selector) SetCount(c uint32)

SetCount sets count of nodes to select from bucket.

func (*Selector) SetFilter 

func (s *Selector) SetFilter(f string)

SetFilter sets filter reference to select from.

func (*Selector) SetName 

func (s *Selector) SetName(name string)

SetName sets selector name.

func (*Selector) ToV2 

func (s *Selector) ToV2() *netmap.Selector

ToV2 converts Selector to v2 Selector.

func (*Selector) Unmarshal 

func (s *Selector) Unmarshal(data []byte) error

Unmarshal unmarshals protobuf binary representation of Selector.

func (*Selector) UnmarshalJSON 

func (s *Selector) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes Selector from protobuf JSON format.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.