ingest

Documentation

Index

• type Ingester
  • func NewIngester(cfg config.Ingest, h host.Host, idxr *indexer.Engine, reg *registry.Registry, ...) (*Ingester, error)
  • func (ing *Ingester) Close() error
  • func (ing *Ingester) Sync(ctx context.Context, peerID peer.ID, peerAddr multiaddr.Multiaddr) (<-chan multihash.Multihash, error)

Types

type Ingester

type Ingester struct {
	// contains filtered or unexported fields
}

Ingester is a type that uses go-legs for the ingestion protocol.

func NewIngester

func NewIngester(cfg config.Ingest, h host.Host, idxr *indexer.Engine, reg *registry.Registry, ds datastore.Batching) (*Ingester, error)

NewIngester creates a new Ingester that uses a go-legs Subscriber to handle communication with providers.

func (*Ingester) Close

func (ing *Ingester) Close() error

func (*Ingester) Sync

func (ing *Ingester) Sync(ctx context.Context, peerID peer.ID, peerAddr multiaddr.Multiaddr) (<-chan multihash.Multihash, error)

Sync syncs the latest advertisement from a publisher. This is done by first fetching the latest advertisement ID from and traversing it until traversal gets to the last seen advertisement. Then the entries in each advertisement are synced and the multihashes in each entry are indexed.

The Context argument controls the lifetime of the sync. Canceling it cancels the sync and causes the multihash channel to close without any data.

Note that the multihash entries corresponding to the advertisement are synced in the background. The completion of advertisement sync does not necessarily mean that the entries corresponding to the advertisement are synced.