config

Documentation

Index

• Constants
• Variables
• func Filename(configRoot string) (string, error)
• func Marshal(value interface{}) ([]byte, error)
• func Path(configRoot, configFile string) (string, error)
• func PathRoot() (string, error)
• type Addresses
  • func NewAddresses() Addresses
• type Bootstrap
  • func NewBootstrap() Bootstrap
  • func (b Bootstrap) PeerAddrs() ([]peer.AddrInfo, error)
  • func (b *Bootstrap) SetPeers(addrs []peer.AddrInfo)
• type Config
  • func Init(out io.Writer) (*Config, error)
  • func InitWithIdentity(identity Identity) (*Config, error)
  • func Load(filePath string) (*Config, error)
  • func (c *Config) Save(filePath string) error
  • func (c *Config) String() string
• type Datastore
  • func NewDatastore() Datastore
• type Discovery
  • func NewDiscovery() Discovery
• type Duration
  • func (d Duration) MarshalText() ([]byte, error)
  • func (d Duration) String() string
  • func (d *Duration) UnmarshalText(text []byte) error
• type Identity
  • func CreateIdentity(out io.Writer) (Identity, error)
  • func (i Identity) Decode() (peer.ID, ic.PrivKey, error)
  • func (i Identity) DecodePrivateKey(passphrase string) (ic.PrivKey, error)
• type Indexer
  • func NewIndexer() Indexer
• type Ingest
  • func NewIngest() Ingest
  • func (c *Ingest) AdvertisementRecursionLimit() selector.RecursionLimit
  • func (c *Ingest) EntriesRecursionLimit() selector.RecursionLimit
• type Policy
  • func NewPolicy() Policy

Constants

const (
	// DefaultPathName is the default config dir name.
	DefaultPathName = ".storetheindex"
	// DefaultPathRoot is the path to the default config dir location.
	DefaultPathRoot = "~/" + DefaultPathName
	// DefaultConfigFile is the filename of the configuration file.
	DefaultConfigFile = "config"
	// EnvDir is the environment variable used to change the path root.
	EnvDir = "STORETHEINDEX_PATH"
)

const (
	IdentityTag     = "Identity"
	PrivKeyTag      = "PrivKey"
	PrivKeySelector = IdentityTag + "." + PrivKeyTag
)

Variables

var (
	ErrInitialized    = errors.New("configuration file already exists")
	ErrNotInitialized = errors.New("not initialized")
)

var ErrInvalidPeerAddr = errors.New("invalid peer address")

ErrInvalidPeerAddr signals an address is not a valid peer address.

Functions

func Filename

func Filename(configRoot string) (string, error)

Filename returns the configuration file path given a configuration root directory. If the configuration root directory is empty, use the default.

func Marshal

func Marshal(value interface{}) ([]byte, error)

Marshal configuration with JSON.

func Path

func Path(configRoot, configFile string) (string, error)

Path returns the config file path relative to the configuration root. If an empty string is provided for `configRoot`, the default root is used. If configFile is an absolute path, then configRoot is ignored.

func PathRoot

func PathRoot() (string, error)

PathRoot returns the default configuration root directory.

Types

type Addresses

type Addresses struct {
	// Admin is the admin http listen address. Set to "none" to disable this
	// server for both http and libp2p.
	Admin string
	// Finder is the finder http isten address. Set to "none" to disable this
	// server for both http and libp2p.
	Finder string
	// Ingest is the index data ingestion http listen address. Set to "none"
	// to disable this server for both http and libp2p.
	Ingest string
	// P2PMaddr is the libp2p host multiaddr for all servers. Set to "none" to
	// disable libp2p hosting.
	P2PAddr string
}

Addresses stores the (string) multiaddr addresses for the node.

func NewAddresses

func NewAddresses() Addresses

NewAddresses returns Addresses with values set to their defaults.

type Bootstrap

type Bootstrap struct {
	// Peers is the local node's bootstrap peer addresses
	Peers []string
	// MinimumPeers governs whether to bootstrap more connections. If the node
	// has less open connections than this number, it will open connections to
	// the bootstrap nodes.  Set to 0 to disable bootstrapping.
	MinimumPeers int
}

Bootstrap configures other nodes to connect to for the purpose of exchanging gossip pubsub. The nodes listed here must be running pubsub and must also be subscribed to the indexer/ingest topic. The peers can be other indexers, or IPFS nodes with pubsub enabled and subscribed to the topic.

func NewBootstrap

func NewBootstrap() Bootstrap

NewBootstrap returns Bootstrap with values set to their defaults.

func (Bootstrap) PeerAddrs

func (b Bootstrap) PeerAddrs() ([]peer.AddrInfo, error)

PeerAddrs returns the bootstrap peers as a list of AddrInfo.

func (*Bootstrap) SetPeers

func (b *Bootstrap) SetPeers(addrs []peer.AddrInfo)

SetPeers sets the bootstrap peers from a list of AddrInfo.

type Config

type Config struct {
	Identity  Identity  // peer identity
	Addresses Addresses // addresses to listen on
	Bootstrap Bootstrap // Peers to connect to for gossip
	Datastore Datastore // datastore config
	Discovery Discovery // provider pubsub peers
	Indexer   Indexer   // indexer code configuration
	Ingest    Ingest    // ingestion related configuration.
}

Config is used to load config files.

func Init

func Init(out io.Writer) (*Config, error)

func InitWithIdentity

func InitWithIdentity(identity Identity) (*Config, error)

func Load

func Load(filePath string) (*Config, error)

Load reads the json-serialized config at the specified path.

func (*Config) Save

func (c *Config) Save(filePath string) error

Save writes the json-serialized config to the specified path.

func (*Config) String

func (c *Config) String() string

String returns a pretty-printed json config.

type Datastore

type Datastore struct {
	// Type is the type of datastore.
	Type string
	// Dir is the directory where the datastore is kept. If this is not an
	// absolute path then the location is relative to the indexer repo
	// directory.
	Dir string
}

Datastore tracks the configuration of the datastore.

func NewDatastore

func NewDatastore() Datastore

NewDatastore returns Datastore with values set to their defaults.

type Discovery

type Discovery struct {
	// LotusGateway is the host or host:port for a lotus gateway used to
	// verify providers on the blockchain.
	LotusGateway string
	// Policy configures which providers are allowed and blocked
	Policy Policy
	// PollInterval is the amount of time to wait without getting any updates
	// from a provider, before sending a request for the latest advertisement.
	// Values are a number ending in "s", "m", "h" for seconds. minutes, hours.
	PollInterval Duration
	// RediscoverWait is the amount of time that must pass before a provider
	// can be discovered following a previous discovery attempt.  A value of 0
	// means there is no wait time.
	RediscoverWait Duration
	// Timeout is the maximum amount of time that the indexer will spend trying
	// to discover and verify a new provider.
	Timeout Duration
}

Discovery holds addresses of peers to from which to receive index advertisements, which peers to allow and block, and time related settings for provider discovery

func NewDiscovery

func NewDiscovery() Discovery

NewDiscovery returns Discovery with values set to their defaults.

type Duration

type Duration time.Duration

Duration wraps time.Duration to provide json serialization and deserialization.

NOTE: the zero value encodes to an empty string.

func (Duration) MarshalText

func (d Duration) MarshalText() ([]byte, error)

func (Duration) String

func (d Duration) String() string

func (*Duration) UnmarshalText

func (d *Duration) UnmarshalText(text []byte) error

type Identity

type Identity struct {
	PeerID  string
	PrivKey string `json:",omitempty"`
}

Identity tracks the configuration of the local node's identity.

func CreateIdentity

func CreateIdentity(out io.Writer) (Identity, error)

CreateIdentity initializes a new identity.

func (Identity) Decode

func (i Identity) Decode() (peer.ID, ic.PrivKey, error)

func (Identity) DecodePrivateKey

func (i Identity) DecodePrivateKey(passphrase string) (ic.PrivKey, error)

DecodePrivateKey is a helper to decode the user's PrivateKey.

type Indexer

type Indexer struct {
	// Maximum number of CIDs that cache can hold. Setting to 0 disables cache.
	CacheSize int
	// Directory where value store is kept. If this is not an absolute path
	// then the location is relative to the indexer repo directory.
	ValueStoreDir string
	// Type of valuestore to use, such as "sti" or "pogreb".
	ValueStoreType string
}

Indexer holds configuration for the indexer core.

func NewIndexer

func NewIndexer() Indexer

NewIndexer returns Indexer with values set to their defaults.

type Ingest

type Ingest struct {
	// PubSubTopic used to advertise ingestion announcements.
	PubSubTopic string
	// StoreBatchSize is the number of entries in each write to the value
	// store.  Specifying a value less than 2 disables batching.  This should
	// be smaller than the maximum number of multihashes in an entry block to
	// write concurrently to the value store.
	StoreBatchSize int
	// SyncTimeout is the maximum amount of time allowed for a sync to complete
	// before it is canceled. This can be a sync of a chain of advertisements
	// or a chain of advertisement entries.  The value is an integer string
	// ending in "s", "m", "h" for seconds. minutes, hours.
	SyncTimeout Duration

	// The recursion depth limit when syncing advertisement entries.
	// The value -1 means no limit. Defaults to 400.
	EntriesDepthLimit int64

	// The recursion depth limit when syncing advertisements.
	// The value -1 means no limit. Defaults to 400.
	AdvertisementDepthLimit int64
}

Ingest tracks the configuration related to the ingestion protocol.

func NewIngest

func NewIngest() Ingest

NewIngest returns Ingest with values set to their defaults.

func (*Ingest) AdvertisementRecursionLimit

func (c *Ingest) AdvertisementRecursionLimit() selector.RecursionLimit

AdvertisementRecursionLimit returns the recursion limit of advertisements. See Ingest.AdvertisementDepthLimit

func (*Ingest) EntriesRecursionLimit

func (c *Ingest) EntriesRecursionLimit() selector.RecursionLimit

EntriesRecursionLimit returns the recursion limit of advertisement entries. See Ingest.EntriesDepthLimit

type Policy

type Policy struct {
	// Allow is either false or true, and determines whether a peer is allowed
	// (true) or is blocked (false), by default.
	Allow bool
	// Except is a list of peer IDs that are exceptions to the allow action.
	// Peers that are allowed by policy or exception must still be verified or
	// trusted in order to register.
	Except []string

	// Trust is either false or true, and determines whether an allowed peer
	// can skip (true) on-chain verification or not (false), by default.
	Trust bool
	// TrustExcept is a list of peer IDs that are exceptions to the trust
	// action.  If Trust is false then all allowed peers must be verified,
	// except those listed here.  If Trust is true, then only the peers
	// listed here require verification.
	TrustExcept []string
}

Policy configures which peers are allowed and blocked, and which allowed peers require verification or are already trusted. Currently, this same policy is applied to both publishers and providers.

Publishers and providers are not the same. Publishers are peers that supply data to the indexer. Providers are the peers that appear in advertisements and are where clients will retrieve content from.

Policy evaluation works like two gates that must be passed in order to be allowed to index content. The first gate is the "allow" gate that determines whether a publisher is allowed or not. The second gate is the "trust" gate that determines whether a publisher is trusted or must be verified on-chain to be authorized to index content.

func NewPolicy

func NewPolicy() Policy

NewPolicy returns Policy with values set to their defaults.