cache

Documentation

Index

• type GossipSubSpamRecordCache
  • func NewGossipSubSpamRecordCache(sizeLimit uint32, logger zerolog.Logger, collector module.HeroCacheMetrics, ...) *GossipSubSpamRecordCache
  • func (a *GossipSubSpamRecordCache) Adjust(peerID peer.ID, updateFn p2p.UpdateFunction) (*p2p.GossipSubSpamRecord, error)
  • func (a *GossipSubSpamRecordCache) Get(peerID peer.ID) (*p2p.GossipSubSpamRecord, error, bool)
  • func (a *GossipSubSpamRecordCache) Has(peerID peer.ID) bool
• type IdentifierSet
  • func (s IdentifierSet) Contains(id flow.Identifier) bool
• type NodeDisallowListingWrapper
  • func NewNodeDisallowListWrapper(identityProvider module.IdentityProvider, db *badger.DB, ...) (*NodeDisallowListingWrapper, error)
  • func (w *NodeDisallowListingWrapper) ByNodeID(identifier flow.Identifier) (*flow.Identity, bool)
  • func (w *NodeDisallowListingWrapper) ByPeerID(p peer.ID) (*flow.Identity, bool)
  • func (w *NodeDisallowListingWrapper) ClearDisallowList() error
  • func (w *NodeDisallowListingWrapper) GetDisallowList() flow.IdentifierList
  • func (w *NodeDisallowListingWrapper) Identities(filter flow.IdentityFilter) flow.IdentityList
  • func (w *NodeDisallowListingWrapper) Update(disallowList flow.IdentifierList) error
• type PreprocessorFunc
• type ProtocolStateIDCache
  • func NewProtocolStateIDCache(logger zerolog.Logger, state protocol.State, ...) (*ProtocolStateIDCache, error)
  • func (p *ProtocolStateIDCache) ByNodeID(flowID flow.Identifier) (*flow.Identity, bool)
  • func (p *ProtocolStateIDCache) ByPeerID(peerID peer.ID) (*flow.Identity, bool)
  • func (p *ProtocolStateIDCache) EpochCommittedPhaseStarted(currentEpochCounter uint64, header *flow.Header)
  • func (p *ProtocolStateIDCache) EpochSetupPhaseStarted(currentEpochCounter uint64, header *flow.Header)
  • func (p *ProtocolStateIDCache) EpochTransition(newEpochCounter uint64, header *flow.Header)
  • func (p *ProtocolStateIDCache) GetFlowID(peerID peer.ID) (flow.Identifier, error)
  • func (p *ProtocolStateIDCache) GetPeerID(flowID flow.Identifier) (peer.ID, error)
  • func (p *ProtocolStateIDCache) Identities(filter flow.IdentityFilter) flow.IdentityList

Types

type GossipSubSpamRecordCache

type GossipSubSpamRecordCache struct {
	// contains filtered or unexported fields
}

GossipSubSpamRecordCache is a cache for storing the gossipsub spam records of peers. It is thread-safe. The spam records of peers is used to calculate the application specific score, which is part of the GossipSub score of a peer. Note that neither of the spam records, application specific score, and GossipSub score are shared publicly with other peers. Rather they are solely used by the current peer to select the peers to which it will connect on a topic mesh.

func NewGossipSubSpamRecordCache

func NewGossipSubSpamRecordCache(sizeLimit uint32,
	logger zerolog.Logger,
	collector module.HeroCacheMetrics,
	initFn func() p2p.GossipSubSpamRecord,
	prFns ...PreprocessorFunc) *GossipSubSpamRecordCache

NewGossipSubSpamRecordCache returns a new HeroCache-based application specific Penalty cache. Args:

sizeLimit: the maximum number of entries that can be stored in the cache.
logger: the logger to be used by the cache.
collector: the metrics collector to be used by the cache.

Returns:

*GossipSubSpamRecordCache: the newly created cache with a HeroCache-based backend.

func (*GossipSubSpamRecordCache) Adjust

func (a *GossipSubSpamRecordCache) Adjust(peerID peer.ID, updateFn p2p.UpdateFunction) (*p2p.GossipSubSpamRecord, error)

Adjust updates the GossipSub spam penalty of a peer in the cache. If the peer does not have a record in the cache, a new record is created. The order of the pre-processing functions is the same as the order in which they were added to the cache. Args: - peerID: the peer ID of the peer in the GossipSub protocol. - updateFn: the update function to be applied to the record. Returns: - *GossipSubSpamRecord: the updated record. - error on failure to update the record. The returned error is irrecoverable and indicates an exception. Note that if any of the pre-processing functions returns an error, the record is reverted to its original state (prior to applying the update function).

func (*GossipSubSpamRecordCache) Get

func (a *GossipSubSpamRecordCache) Get(peerID peer.ID) (*p2p.GossipSubSpamRecord, error, bool)

Get returns the spam record of a peer from the cache. Args:

-peerID: the peer ID of the peer in the GossipSub protocol.

Returns:

• the application specific score record of the peer.
• error if the underlying cache update fails, or any of the pre-processors fails. The error is considered irrecoverable, and the caller is advised to crash the node.
• true if the record is found in the cache, false otherwise.

func (*GossipSubSpamRecordCache) Has

func (a *GossipSubSpamRecordCache) Has(peerID peer.ID) bool

Has returns true if the spam record of a peer is found in the cache, false otherwise. Args: - peerID: the peer ID of the peer in the GossipSub protocol. Returns: - true if the gossipsub spam record of the peer is found in the cache, false otherwise.

type IdentifierSet

type IdentifierSet map[flow.Identifier]struct{}

IdentifierSet represents a set of node IDs (operator-defined) whose communication should be blocked.

func (IdentifierSet) Contains

func (s IdentifierSet) Contains(id flow.Identifier) bool

Contains returns true iff id ∈ s

type NodeDisallowListingWrapper

type NodeDisallowListingWrapper struct {
	// contains filtered or unexported fields
}

NodeDisallowListingWrapper is a wrapper for an `module.IdentityProvider` instance, where the wrapper overrides the `Ejected` flag to true for all NodeIDs in a `disallowList`. To avoid modifying the source of the identities, the wrapper creates shallow copies of the identities (whenever necessary) and modifies the `Ejected` flag only in the copy. The `NodeDisallowListingWrapper` internally represents the `disallowList` as a map, to enable performant lookup. However, the exported API works with `flow.IdentifierList` for disallowList, as this is a broadly supported data structure which lends itself better to config or command-line inputs. When a node is disallow-listed, the networking layer connection to that node is closed and no incoming or outgoing connections are established with that node. TODO: terminology change - rename `blocklist` to `disallowList` everywhere to be consistent with the code.

func NewNodeDisallowListWrapper

func NewNodeDisallowListWrapper(
	identityProvider module.IdentityProvider,
	db *badger.DB,
	updateConsumerOracle func() network.DisallowListNotificationConsumer) (*NodeDisallowListingWrapper, error)

NewNodeDisallowListWrapper wraps the given `IdentityProvider`. The disallow-list is loaded from the database (or assumed to be empty if no database entry is present).

func (*NodeDisallowListingWrapper) ByNodeID

func (w *NodeDisallowListingWrapper) ByNodeID(identifier flow.Identifier) (*flow.Identity, bool)

ByNodeID returns the full identity for the node with the given Identifier, where Identifier is the way the protocol refers to the node. The function has the same semantics as a map lookup, where the boolean return value is true if and only if Identity has been found, i.e. `Identity` is not nil. Caution: function returns include ejected nodes. Please check the `Ejected` flag in the identity.

func (*NodeDisallowListingWrapper) ByPeerID

func (w *NodeDisallowListingWrapper) ByPeerID(p peer.ID) (*flow.Identity, bool)

ByPeerID returns the full identity for the node with the given peer ID, peer.ID is the libp2p-level identifier of a Flow node. The function has the same semantics as a map lookup, where the boolean return value is true if and only if Identity has been found, i.e. `Identity` is not nil. Caution: function returns include ejected nodes. Please check the `Ejected` flag in the identity.

func (*NodeDisallowListingWrapper) ClearDisallowList

func (w *NodeDisallowListingWrapper) ClearDisallowList() error

ClearDisallowList purges the set of blocked node IDs. Convenience function equivalent to w.Update(nil). No errors are expected during normal operations.

func (*NodeDisallowListingWrapper) GetDisallowList

func (w *NodeDisallowListingWrapper) GetDisallowList() flow.IdentifierList

GetDisallowList returns the set of blocked node IDs.

func (*NodeDisallowListingWrapper) Identities

func (w *NodeDisallowListingWrapper) Identities(filter flow.IdentityFilter) flow.IdentityList

Identities returns the full identities of _all_ nodes currently known to the protocol that pass the provided filter. Caution, this includes ejected nodes. Please check the `Ejected` flag in the returned identities (or provide a filter for removing ejected nodes).

func (*NodeDisallowListingWrapper) Update

func (w *NodeDisallowListingWrapper) Update(disallowList flow.IdentifierList) error

Update sets the wrapper's internal set of blocked nodes to `disallowList`. Empty list and `nil` (equivalent to empty list) are accepted inputs. To avoid legacy entries in the database, this function purges the entire data base entry if `disallowList` is empty. This implementation is _eventually consistent_, where changes are written to the database first and then (non-atomically!) the in-memory set of blocked nodes is updated. This strongly benefits performance and modularity. No errors are expected during normal operations.

Args: - disallowList: list of node IDs to be disallow-listed from the networking layer, i.e., the existing connections to these nodes will be closed and no new connections will be established (neither incoming nor outgoing).

Returns: - error: if the update fails, e.g., due to a database error. Any returned error is irrecoverable and the caller should abort the process.

type PreprocessorFunc

type PreprocessorFunc func(record p2p.GossipSubSpamRecord, lastUpdated time.Time) (p2p.GossipSubSpamRecord, error)

PreprocessorFunc is a function that is called by the cache upon reading or updating a record in the cache. It is used to perform any necessary pre-processing on the record before returning it when reading or changing it when updating. The effect of the pre-processing is that the record is updated in the cache. If there are multiple pre-processors, they are called in the order they are added to the cache. Args:

record: the record to be pre-processed.
lastUpdated: the last time the record was updated.

Returns:

	GossipSubSpamRecord: the pre-processed record.
 error: an error if the pre-processing failed. The error is considered irrecoverable (unless the parameters can be adjusted and the pre-processing can be retried). The caller is
 advised to crash the node upon an error if failure to read or update the record is not acceptable.

type ProtocolStateIDCache

type ProtocolStateIDCache struct {
	events.Noop
	// contains filtered or unexported fields
}

ProtocolStateIDCache implements an `id.IdentityProvider` and `p2p.IDTranslator` for the set of authorized Flow network participants as according to the given `protocol.State`. the implementation assumes that the node information changes rarely, while queries are frequent. Hence, we follow an event-driven design, where the ProtocolStateIDCache subscribes to relevant protocol notifications (mainly Epoch notifications) and updates its internally cached list of authorized node identities. Note: this implementation is _eventually consistent_, where changes in the protocol state will quickly, but not atomically, propagate to the ProtocolStateIDCache. This strongly benefits performance and modularity, as we can cache identities locally here, while the marginal delay of updates is of no concern to the protocol.

func NewProtocolStateIDCache

func NewProtocolStateIDCache(
	logger zerolog.Logger,
	state protocol.State,
	eventDistributor *events.Distributor,
) (*ProtocolStateIDCache, error)

func (*ProtocolStateIDCache) ByNodeID

func (p *ProtocolStateIDCache) ByNodeID(flowID flow.Identifier) (*flow.Identity, bool)

ByNodeID returns the full identity for the node with the given Identifier, where Identifier is the way the protocol refers to the node. The function has the same semantics as a map lookup, where the boolean return value is true if and only if Identity has been found, i.e. `Identity` is not nil. Caution: function returns include ejected nodes. Please check the `Ejected` flag in the identity.

func (*ProtocolStateIDCache) ByPeerID

func (p *ProtocolStateIDCache) ByPeerID(peerID peer.ID) (*flow.Identity, bool)

ByPeerID returns the full identity for the node with the given peer ID, where ID is the way the libP2P refers to the node. The function has the same semantics as a map lookup, where the boolean return value is true if and only if Identity has been found, i.e. `Identity` is not nil. Caution: function returns include ejected nodes. Please check the `Ejected` flag in the identity.

func (*ProtocolStateIDCache) EpochCommittedPhaseStarted

func (p *ProtocolStateIDCache) EpochCommittedPhaseStarted(currentEpochCounter uint64, header *flow.Header)

EpochCommittedPhaseStarted is a callback function for notifying the `ProtocolStateIDCache` that the EpochCommitted Phase has just stared. Upon such notification, the internally-cached Identity table of authorized network participants is updated.

TODO: per API contract, implementations of `EpochCommittedPhaseStarted` should be non-blocking and virtually latency free. However, we run data base queries and acquire locks here, which is undesired.

func (*ProtocolStateIDCache) EpochSetupPhaseStarted

func (p *ProtocolStateIDCache) EpochSetupPhaseStarted(currentEpochCounter uint64, header *flow.Header)

EpochSetupPhaseStarted is a callback function for notifying the `ProtocolStateIDCache` that the EpochSetup Phase has just stared. Upon such notification, the internally-cached Identity table of authorized network participants is updated.

TODO: per API contract, implementations of `EpochSetupPhaseStarted` should be non-blocking and virtually latency free. However, we run data base queries and acquire locks here, which is undesired.

func (*ProtocolStateIDCache) EpochTransition

func (p *ProtocolStateIDCache) EpochTransition(newEpochCounter uint64, header *flow.Header)

EpochTransition is a callback function for notifying the `ProtocolStateIDCache` of an Epoch transition that just occurred. Upon such notification, the internally-cached Identity table of authorized network participants is updated.

TODO: per API contract, implementations of `EpochTransition` should be non-blocking and virtually latency free. However, we run data base queries and acquire locks here, which is undesired.

func (*ProtocolStateIDCache) GetFlowID

func (p *ProtocolStateIDCache) GetFlowID(peerID peer.ID) (flow.Identifier, error)

GetFlowID returns the Flow ID for the given peer ID. During normal operations, the following error returns are expected

• ErrUnknownId if the given Identifier is unknown

func (*ProtocolStateIDCache) GetPeerID

func (p *ProtocolStateIDCache) GetPeerID(flowID flow.Identifier) (peer.ID, error)

GetPeerID returns the peer ID for the given Flow ID. During normal operations, the following error returns are expected

• ErrUnknownId if the given Identifier is unknown

func (*ProtocolStateIDCache) Identities

func (p *ProtocolStateIDCache) Identities(filter flow.IdentityFilter) flow.IdentityList

Identities returns the full identities of _all_ nodes currently known to the protocol that pass the provided filter. Caution, this includes ejected nodes. Please check the `Ejected` flag in the identities (or provide a filter for removing ejected nodes).