cache

Documentation

Index

• type GossipSubSpamRecordCache
  • func NewGossipSubSpamRecordCache(sizeLimit uint32, logger zerolog.Logger, collector module.HeroCacheMetrics, ...) *GossipSubSpamRecordCache
  • func (a *GossipSubSpamRecordCache) Add(peerId peer.ID, record p2p.GossipSubSpamRecord) bool
  • func (a *GossipSubSpamRecordCache) Get(peerID peer.ID) (*p2p.GossipSubSpamRecord, error, bool)
  • func (a *GossipSubSpamRecordCache) Has(peerID peer.ID) bool
  • func (a *GossipSubSpamRecordCache) Update(peerID peer.ID, updateFn p2p.UpdateFunction) (*p2p.GossipSubSpamRecord, error)
• type IdentifierSet
  • func (s IdentifierSet) Contains(id flow.Identifier) bool
• type NodeBlocklistWrapper
  • func NewNodeBlocklistWrapper(identityProvider module.IdentityProvider, db *badger.DB, ...) (*NodeBlocklistWrapper, error)
  • func (w *NodeBlocklistWrapper) ByNodeID(identifier flow.Identifier) (*flow.Identity, bool)
  • func (w *NodeBlocklistWrapper) ByPeerID(p peer.ID) (*flow.Identity, bool)
  • func (w *NodeBlocklistWrapper) ClearBlocklist() error
  • func (w *NodeBlocklistWrapper) GetBlocklist() flow.IdentifierList
  • func (w *NodeBlocklistWrapper) Identities(filter flow.IdentityFilter) flow.IdentityList
  • func (w *NodeBlocklistWrapper) Update(blocklist flow.IdentifierList) error
• type PreprocessorFunc
• type ProtocolStateIDCache
  • func NewProtocolStateIDCache(logger zerolog.Logger, state protocol.State, ...) (*ProtocolStateIDCache, error)
  • func (p *ProtocolStateIDCache) ByNodeID(flowID flow.Identifier) (*flow.Identity, bool)
  • func (p *ProtocolStateIDCache) ByPeerID(peerID peer.ID) (*flow.Identity, bool)
  • func (p *ProtocolStateIDCache) EpochCommittedPhaseStarted(currentEpochCounter uint64, header *flow.Header)
  • func (p *ProtocolStateIDCache) EpochSetupPhaseStarted(currentEpochCounter uint64, header *flow.Header)
  • func (p *ProtocolStateIDCache) EpochTransition(newEpochCounter uint64, header *flow.Header)
  • func (p *ProtocolStateIDCache) GetFlowID(peerID peer.ID) (flow.Identifier, error)
  • func (p *ProtocolStateIDCache) GetPeerID(flowID flow.Identifier) (peer.ID, error)
  • func (p *ProtocolStateIDCache) Identities(filter flow.IdentityFilter) flow.IdentityList

Types

type GossipSubSpamRecordCache

type GossipSubSpamRecordCache struct {
	// contains filtered or unexported fields
}

GossipSubSpamRecordCache is a cache for storing the gossipsub spam records of peers. It is thread-safe. The spam records of peers is used to calculate the application specific score, which is part of the GossipSub score of a peer. Note that neither of the spam records, application specific score, and GossipSub score are shared publicly with other peers. Rather they are solely used by the current peer to select the peers to which it will connect on a topic mesh.

func NewGossipSubSpamRecordCache

func NewGossipSubSpamRecordCache(sizeLimit uint32, logger zerolog.Logger, collector module.HeroCacheMetrics, prFns ...PreprocessorFunc) *GossipSubSpamRecordCache

NewGossipSubSpamRecordCache returns a new HeroCache-based application specific Penalty cache. Args:

sizeLimit: the maximum number of entries that can be stored in the cache.
logger: the logger to be used by the cache.
collector: the metrics collector to be used by the cache.

Returns:

*GossipSubSpamRecordCache: the newly created cache with a HeroCache-based backend.

func (*GossipSubSpamRecordCache) Add

func (a *GossipSubSpamRecordCache) Add(peerId peer.ID, record p2p.GossipSubSpamRecord) bool

Add adds the GossipSubSpamRecord of a peer to the cache. Args: - peerID: the peer ID of the peer in the GossipSub protocol. - record: the GossipSubSpamRecord of the peer.

Returns: - bool: true if the record was added successfully, false otherwise. Note that a record is added successfully if the cache has enough space to store the record and no record exists for the peer in the cache. In other words, the entries are deduplicated by the peer ID.

func (*GossipSubSpamRecordCache) Get

func (a *GossipSubSpamRecordCache) Get(peerID peer.ID) (*p2p.GossipSubSpamRecord, error, bool)

Get returns the spam record of a peer from the cache. Args:

-peerID: the peer ID of the peer in the GossipSub protocol.

Returns:

• the application specific score record of the peer.
• error if the underlying cache update fails, or any of the pre-processors fails. The error is considered irrecoverable, and the caller is advised to crash the node.
• true if the record is found in the cache, false otherwise.

func (*GossipSubSpamRecordCache) Has

func (a *GossipSubSpamRecordCache) Has(peerID peer.ID) bool

Has returns true if the spam record of a peer is found in the cache, false otherwise. Args: - peerID: the peer ID of the peer in the GossipSub protocol. Returns: - true if the gossipsub spam record of the peer is found in the cache, false otherwise.

func (*GossipSubSpamRecordCache) Update

func (a *GossipSubSpamRecordCache) Update(peerID peer.ID, updateFn p2p.UpdateFunction) (*p2p.GossipSubSpamRecord, error)

Update updates the GossipSub spam penalty of a peer in the cache. It assumes that a record already exists for the peer in the cache. It first reads the record from the cache, applies the pre-processing functions to the record, and then applies the update function to the record. The order of the pre-processing functions is the same as the order in which they were added to the cache. Args: - peerID: the peer ID of the peer in the GossipSub protocol. - updateFn: the update function to be applied to the record. Returns: - *GossipSubSpamRecord: the updated record. - error on failure to update the record. The returned error is irrecoverable and indicates an exception. Note that if any of the pre-processing functions returns an error, the record is reverted to its original state (prior to applying the update function).

type IdentifierSet

type IdentifierSet map[flow.Identifier]struct{}

IdentifierSet represents a set of node IDs (operator-defined) whose communication should be blocked.

func (IdentifierSet) Contains

func (s IdentifierSet) Contains(id flow.Identifier) bool

Contains returns true iff id ∈ s

type NodeBlocklistWrapper

type NodeBlocklistWrapper struct {
	// contains filtered or unexported fields
}

NodeBlocklistWrapper is a wrapper for an `module.IdentityProvider` instance, where the wrapper overrides the `Ejected` flag to true for all NodeIDs in a `blocklist`. To avoid modifying the source of the identities, the wrapper creates shallow copies of the identities (whenever necessary) and modifies the `Ejected` flag only in the copy. The `NodeBlocklistWrapper` internally represents the `blocklist` as a map, to enable performant lookup. However, the exported API works with `flow.IdentifierList` for blocklist, as this is a broadly supported data structure which lends itself better to config or command-line inputs.

func NewNodeBlocklistWrapper

func NewNodeBlocklistWrapper(
	identityProvider module.IdentityProvider,
	db *badger.DB,
	distributor p2p.DisallowListNotificationDistributor) (*NodeBlocklistWrapper, error)

NewNodeBlocklistWrapper wraps the given `IdentityProvider`. The blocklist is loaded from the database (or assumed to be empty if no database entry is present).

func (*NodeBlocklistWrapper) ByNodeID

func (w *NodeBlocklistWrapper) ByNodeID(identifier flow.Identifier) (*flow.Identity, bool)

ByNodeID returns the full identity for the node with the given Identifier, where Identifier is the way the protocol refers to the node. The function has the same semantics as a map lookup, where the boolean return value is true if and only if Identity has been found, i.e. `Identity` is not nil. Caution: function returns include ejected nodes. Please check the `Ejected` flag in the identity.

func (*NodeBlocklistWrapper) ByPeerID

func (w *NodeBlocklistWrapper) ByPeerID(p peer.ID) (*flow.Identity, bool)

ByPeerID returns the full identity for the node with the given peer ID, peer.ID is the libp2p-level identifier of a Flow node. The function has the same semantics as a map lookup, where the boolean return value is true if and only if Identity has been found, i.e. `Identity` is not nil. Caution: function returns include ejected nodes. Please check the `Ejected` flag in the identity.

func (*NodeBlocklistWrapper) ClearBlocklist

func (w *NodeBlocklistWrapper) ClearBlocklist() error

ClearBlocklist purges the set of blocked node IDs. Convenience function equivalent to w.Update(nil). No errors are expected during normal operations.

func (*NodeBlocklistWrapper) GetBlocklist

func (w *NodeBlocklistWrapper) GetBlocklist() flow.IdentifierList

GetBlocklist returns the set of blocked node IDs.

func (*NodeBlocklistWrapper) Identities

func (w *NodeBlocklistWrapper) Identities(filter flow.IdentityFilter) flow.IdentityList

Identities returns the full identities of _all_ nodes currently known to the protocol that pass the provided filter. Caution, this includes ejected nodes. Please check the `Ejected` flag in the returned identities (or provide a filter for removing ejected nodes).

func (*NodeBlocklistWrapper) Update

func (w *NodeBlocklistWrapper) Update(blocklist flow.IdentifierList) error

Update sets the wrapper's internal set of blocked nodes to `blocklist`. Empty list and `nil` (equivalent to empty list) are accepted inputs. To avoid legacy entries in the data base, this function purges the entire data base entry if `blocklist` is empty. This implementation is _eventually consistent_, where changes are written to the data base first and then (non-atomically!) the in-memory set of blocked nodes is updated. This strongly benefits performance and modularity. No errors are expected during normal operations.

type PreprocessorFunc

type PreprocessorFunc func(record p2p.GossipSubSpamRecord, lastUpdated time.Time) (p2p.GossipSubSpamRecord, error)

PreprocessorFunc is a function that is called by the cache upon reading or updating a record in the cache. It is used to perform any necessary pre-processing on the record before returning it when reading or changing it when updating. The effect of the pre-processing is that the record is updated in the cache. If there are multiple pre-processors, they are called in the order they are added to the cache. Args:

record: the record to be pre-processed.
lastUpdated: the last time the record was updated.

Returns:

	GossipSubSpamRecord: the pre-processed record.
 error: an error if the pre-processing failed. The error is considered irrecoverable (unless the parameters can be adjusted and the pre-processing can be retried). The caller is
 advised to crash the node upon an error if failure to read or update the record is not acceptable.

type ProtocolStateIDCache

type ProtocolStateIDCache struct {
	events.Noop
	// contains filtered or unexported fields
}

ProtocolStateIDCache implements an `id.IdentityProvider` and `p2p.IDTranslator` for the set of authorized Flow network participants as according to the given `protocol.State`. the implementation assumes that the node information changes rarely, while queries are frequent. Hence, we follow an event-driven design, where the ProtocolStateIDCache subscribes to relevant protocol notifications (mainly Epoch notifications) and updates its internally cached list of authorized node identities. Note: this implementation is _eventually consistent_, where changes in the protocol state will quickly, but not atomically, propagate to the ProtocolStateIDCache. This strongly benefits performance and modularity, as we can cache identities locally here, while the marginal delay of updates is of no concern to the protocol.

func NewProtocolStateIDCache

func NewProtocolStateIDCache(
	logger zerolog.Logger,
	state protocol.State,
	eventDistributor *events.Distributor,
) (*ProtocolStateIDCache, error)

func (*ProtocolStateIDCache) ByNodeID

func (p *ProtocolStateIDCache) ByNodeID(flowID flow.Identifier) (*flow.Identity, bool)

ByNodeID returns the full identity for the node with the given Identifier, where Identifier is the way the protocol refers to the node. The function has the same semantics as a map lookup, where the boolean return value is true if and only if Identity has been found, i.e. `Identity` is not nil. Caution: function returns include ejected nodes. Please check the `Ejected` flag in the identity.

func (*ProtocolStateIDCache) ByPeerID

func (p *ProtocolStateIDCache) ByPeerID(peerID peer.ID) (*flow.Identity, bool)

ByPeerID returns the full identity for the node with the given peer ID, where ID is the way the libP2P refers to the node. The function has the same semantics as a map lookup, where the boolean return value is true if and only if Identity has been found, i.e. `Identity` is not nil. Caution: function returns include ejected nodes. Please check the `Ejected` flag in the identity.

func (*ProtocolStateIDCache) EpochCommittedPhaseStarted

func (p *ProtocolStateIDCache) EpochCommittedPhaseStarted(currentEpochCounter uint64, header *flow.Header)

EpochCommittedPhaseStarted is a callback function for notifying the `ProtocolStateIDCache` that the EpochCommitted Phase has just stared. Upon such notification, the internally-cached Identity table of authorized network participants is updated.

TODO: per API contract, implementations of `EpochCommittedPhaseStarted` should be non-blocking and virtually latency free. However, we run data base queries and acquire locks here, which is undesired.

func (*ProtocolStateIDCache) EpochSetupPhaseStarted

func (p *ProtocolStateIDCache) EpochSetupPhaseStarted(currentEpochCounter uint64, header *flow.Header)

EpochSetupPhaseStarted is a callback function for notifying the `ProtocolStateIDCache` that the EpochSetup Phase has just stared. Upon such notification, the internally-cached Identity table of authorized network participants is updated.

TODO: per API contract, implementations of `EpochSetupPhaseStarted` should be non-blocking and virtually latency free. However, we run data base queries and acquire locks here, which is undesired.

func (*ProtocolStateIDCache) EpochTransition

func (p *ProtocolStateIDCache) EpochTransition(newEpochCounter uint64, header *flow.Header)

EpochTransition is a callback function for notifying the `ProtocolStateIDCache` of an Epoch transition that just occurred. Upon such notification, the internally-cached Identity table of authorized network participants is updated.

TODO: per API contract, implementations of `EpochTransition` should be non-blocking and virtually latency free. However, we run data base queries and acquire locks here, which is undesired.

func (*ProtocolStateIDCache) GetFlowID

func (p *ProtocolStateIDCache) GetFlowID(peerID peer.ID) (flow.Identifier, error)

GetFlowID returns the Flow ID for the given peer ID. During normal operations, the following error returns are expected

• ErrUnknownId if the given Identifier is unknown

func (*ProtocolStateIDCache) GetPeerID

func (p *ProtocolStateIDCache) GetPeerID(flowID flow.Identifier) (peer.ID, error)

GetPeerID returns the peer ID for the given Flow ID. During normal operations, the following error returns are expected

• ErrUnknownId if the given Identifier is unknown

func (*ProtocolStateIDCache) Identities

func (p *ProtocolStateIDCache) Identities(filter flow.IdentityFilter) flow.IdentityList

Identities returns the full identities of _all_ nodes currently known to the protocol that pass the provided filter. Caution, this includes ejected nodes. Please check the `Ejected` flag in the identities (or provide a filter for removing ejected nodes).