config

package
v0.0.0-...-403c8b4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 18, 2022 License: Apache-2.0, MIT Imports: 14 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// DefaultPathName is the default config dir name.
	DefaultPathName = ".storetheindex"
	// DefaultPathRoot is the path to the default config dir location.
	DefaultPathRoot = "~/" + DefaultPathName
	// DefaultConfigFile is the filename of the configuration file.
	DefaultConfigFile = "config"
	// EnvDir is the environment variable used to change the path root.
	EnvDir = "STORETHEINDEX_PATH"

	Version = 2
)
View Source 
const (
	IdentityTag     = "Identity"
	PrivKeyTag      = "PrivKey"
	PrivKeySelector = IdentityTag + "." + PrivKeyTag

	PrivateKeyPathEnvVar = "STORETHEINDEX_PRIV_KEY_PATH"
)

Variables

View Source 
var (
	ErrInitialized    = errors.New("configuration file already exists")
	ErrNotInitialized = errors.New("not initialized")
)
View Source 
var ErrInvalidPeerAddr = errors.New("invalid peer address")

ErrInvalidPeerAddr signals an address is not a valid peer address.

Functions

func Filename 

func Filename(configRoot string) (string, error)

Filename returns the configuration file path given a configuration root directory. If the configuration root directory is empty, use the default.

func Marshal 

func Marshal(value interface{}) ([]byte, error)

Marshal configuration with JSON.

func Path 

func Path(configRoot, configFile string) (string, error)

Path returns the config file path relative to the configuration root. If an empty string is provided for `configRoot`, the default root is used. If configFile is an absolute path, then configRoot is ignored.

func PathRoot 

func PathRoot() (string, error)

PathRoot returns the default configuration root directory.

Types

type Addresses 

type Addresses struct {
	// Admin is the admin http listen address. Set to "none" to disable this
	// server for both http and libp2p.
	Admin string
	// Finder is the finder http isten address. Set to "none" to disable this
	// server for both http and libp2p.
	Finder string
	// FinderWebpage is a domain to display when the homepage of the finder
	// is accessed over http.
	FinderWebpage string
	// Ingest is the index data ingestion http listen address. Set to "none"
	// to disable this server for both http and libp2p.
	Ingest string
	// P2PMaddr is the libp2p host multiaddr for all servers. Set to "none" to
	// disable libp2p hosting.
	P2PAddr string
	// NoResourceManager disables the libp2p resource manager when true.
	NoResourceManager bool
}

Addresses stores the (string) multiaddr addresses for the node.

func NewAddresses 

func NewAddresses() Addresses

NewAddresses returns Addresses with values set to their defaults.

type Bootstrap 

type Bootstrap struct {
	// Peers is the local node's bootstrap peer addresses.
	Peers []string
	// MinimumPeers governs whether to bootstrap more connections. If the node
	// has less open connections than this number, it will open connections to
	// the bootstrap nodes. Set to 0 to disable bootstrapping.
	MinimumPeers int
}

Bootstrap configures other nodes to connect to for the purpose of exchanging gossip pubsub. The nodes listed here must be running pubsub and must also be subscribed to the indexer/ingest topic. The peers can be other indexers, or IPFS nodes with pubsub enabled and subscribed to the topic.

func NewBootstrap 

func NewBootstrap() Bootstrap

NewBootstrap returns Bootstrap with values set to their defaults.

func (Bootstrap) PeerAddrs 

func (b Bootstrap) PeerAddrs() ([]peer.AddrInfo, error)

PeerAddrs returns the bootstrap peers as a list of AddrInfo.

func (*Bootstrap) SetPeers 

func (b *Bootstrap) SetPeers(addrs []peer.AddrInfo)

SetPeers sets the bootstrap peers from a list of AddrInfo.

type Config 

type Config struct {
	Version   int       // config version
	Identity  Identity  // peer identity
	Addresses Addresses // addresses to listen on
	Bootstrap Bootstrap // Peers to connect to for gossip
	Datastore Datastore // datastore config
	Discovery Discovery // provider pubsub peers
	Indexer   Indexer   // indexer code configuration
	Ingest    Ingest    // ingestion related configuration.
	Logging   Logging   // logging configuration.
	Peering   Peering   // peering service configuration.
}

Config is used to load config files.

func Init 

func Init(out io.Writer) (*Config, error)

func InitWithIdentity 

func InitWithIdentity(identity Identity) (*Config, error)

func Load 

func Load(filePath string) (*Config, error)

Load reads the json-serialized config at the specified path.

func (*Config) Save 

func (c *Config) Save(filePath string) error

Save writes the json-serialized config to the specified path.

func (*Config) String 

func (c *Config) String() string

String returns a pretty-printed json config.

func (*Config) UpgradeConfig 

func (c *Config) UpgradeConfig(filePath string) error

UpgradeConfig upgrades (or downgrades) the config file to the current version. If the config file is at the current version a backup is still created and the config rewritten with any unconfigured values set to their defaults.

type Datastore 

type Datastore struct {
	// Dir is the directory where the datastore is kept. If this is not an
	// absolute path then the location is relative to the indexer repo
	// directory.
	Dir string
	// Type is the type of datastore.
	Type string
}

Datastore tracks the configuration of the datastore.

func NewDatastore 

func NewDatastore() Datastore

NewDatastore returns Datastore with values set to their defaults.

type Discovery 

type Discovery struct {
	// LotusGateway is the host or host:port for a lotus gateway used to
	// verify providers on the blockchain.
	LotusGateway string
	// Policy configures which providers are allowed and blocked, rate-limited,
	// and allow to publish on behalf of others.
	Policy Policy
	// PollInterval is the amount of time to wait without getting any updates
	// for a provider, before sending a request for the latest advertisement.
	// Values are a number ending in "s", "m", "h" for seconds. minutes, hours.
	PollInterval Duration
	// PollRetryAfter is the amount of time from one poll attempt, without a
	// response, to the next poll attempt, and is also the time between checks
	// for providers to poll. This value must be smaller than PollStopAfter for
	// there to be more than one poll attempt for a provider.
	PollRetryAfter Duration
	// PollStopAfter is the amount of time, from the start of polling, to
	// continuing polling for the latest advertisment without getting a
	// response.
	PollStopAfter Duration
	// PollOverrides configures polling for specific providers.
	PollOverrides []Polling
	// RediscoverWait is the amount of time that must pass before a provider
	// can be discovered following a previous discovery attempt. A value of 0
	// means there is no wait time.
	RediscoverWait Duration
	// Timeout is the maximum amount of time that the indexer will spend trying
	// to discover and verify a new provider.
	Timeout Duration
}

Discovery holds addresses of peers to from which to receive index advertisements, which peers to allow and block, and time related settings for provider discovery

func NewDiscovery 

func NewDiscovery() Discovery

NewDiscovery returns Discovery with values set to their defaults.

type Duration 

type Duration time.Duration

Duration wraps time.Duration to provide json serialization and deserialization.

NOTE: the zero value encodes to an empty string.

func (Duration) MarshalText 

func (d Duration) MarshalText() ([]byte, error)

func (Duration) String 

func (d Duration) String() string

func (*Duration) UnmarshalText 

func (d *Duration) UnmarshalText(text []byte) error

type Identity 

type Identity struct {
	// PeerID is the peer ID of the indexer that must match the given PrivKey.
	// If unspecified, it is automatically generated from the PrivKey.
	PeerID string
	// PrivKey represents the peer identity of the indexer.
	//
	// If unset, the key is loaded from the file at the path specified via
	// STORETHEINDEX_PRIV_KEY_PATH environment variable.
	PrivKey string `json:",omitempty"`
}

Identity tracks the configuration of the local node's identity.

func CreateIdentity 

func CreateIdentity(out io.Writer) (Identity, error)

CreateIdentity initializes a new identity.

func (Identity) Decode 

func (i Identity) Decode() (peer.ID, ic.PrivKey, error)

func (Identity) DecodePrivateKey 

func (i Identity) DecodePrivateKey(passphrase string) (ic.PrivKey, error)

DecodePrivateKey is a helper to decode the user's PrivateKey.

type Indexer 

type Indexer struct {
	// Maximum number of CIDs that cache can hold. Setting to -1 disables the
	// cache.
	CacheSize int
	// ConfigCheckInterval is the time between config file update checks.
	ConfigCheckInterval Duration
	// GCInterval configures the garbage collection interval for valuestores
	// that support it.
	GCInterval Duration
	// ShutdownTimeout is the duration that a graceful shutdown has to complete
	// before the daemon process is terminated.
	ShutdownTimeout Duration
	// Directory where value store is kept. If this is not an absolute path
	// then the location is relative to the indexer repo directory.
	ValueStoreDir string
	// Type of valuestore to use, such as "sth" or "pogreb".
	ValueStoreType string
}

Indexer holds configuration for the indexer core. Setting any of these items to their zero-value configures the default value.

func NewIndexer 

func NewIndexer() Indexer

NewIndexer returns Indexer with values set to their defaults.

type Ingest 

type Ingest struct {
	// AdvertisementDepthLimit is the total maximum recursion depth limit when
	// syncing advertisements. The value -1 means no limit and zero means use
	// the default value. Limiting the depth of advertisements can be done if
	// there is a need to prevent an indexer from ingesting long chains of
	// advertisements.
	//
	// A chain of advertisements is synced by separate requests to the provider
	// for each advertisement. These requests are done in groups (segments) of
	// size set by SyncSegmentDepthLimit. AdvertisementDepthLimit sets the
	// limit on the total number of advertisements across all segments.
	AdvertisementDepthLimit int
	// EntriesDepthLimit is the total maximum recursion depth limit when
	// syncing advertisement entries. The value -1 means no limit and zero
	// means use the default value. The purpose is to prevent overload from
	// extremely long entry chains resulting from publisher misconfiguration.
	//
	// A chain of multihash entries chunks is synced by separate requests to
	// the provider for each chunk. These requests are done in groups
	// (segments) of size set by SyncSegmentDepthLimit. EntriesDepthLimit sets
	// the limit on the total number of entries chunks across all segments.
	EntriesDepthLimit int
	// HttpSyncRetryMax sets the maximum number of times HTTP sync requests
	// should be retried.
	HttpSyncRetryMax int
	// HttpSyncRetryWaitMax sets the maximum time to wait before retrying a
	// failed HTTP sync.
	HttpSyncRetryWaitMax Duration
	// HttpSyncRetryWaitMin sets the minimum time to wait before retrying a
	// failed HTTP sync.
	HttpSyncRetryWaitMin Duration
	// HttpSyncTimeout sets the time limit for HTTP sync requests.
	HttpSyncTimeout Duration
	// IngestWorkerCount sets how many ingest worker goroutines to spawn. This
	// controls how many concurrent ingest from different providers we can handle.
	IngestWorkerCount int
	// PubSubTopic sets the topic name to which to subscribe for ingestion
	// announcements.
	PubSubTopic string
	// RateLimit contains rate-limiting configuration.
	RateLimit RateLimit
	// ResendDirectAnnounce determines whether or not to re-publish direct
	// announce messages over gossip pubsub. When a single indexer receives an
	// announce message via HTTP, enabling this lets the indexers re-publish
	// the announce so that other indexers can also receive it.
	ResendDirectAnnounce bool
	// StoreBatchSize is the number of entries in each write to the value
	// store. Specifying a value less than 2 disables batching. This should be
	// smaller than the maximum number of multihashes in an entry block to
	// write concurrently to the value store.
	StoreBatchSize int
	// SyncSegmentDepthLimit is the depth limit of a single sync in a series of
	// calls that collectively sync advertisements or their entries. The value
	// -1 disables the segmentation where the sync will be done in a single call
	// and zero means use the default value.
	SyncSegmentDepthLimit int
	// SyncTimeout is the maximum amount of time allowed for a sync to complete
	// before it is canceled. This can be a sync of a chain of advertisements
	// or a chain of advertisement entries. The value is an integer string
	// ending in "s", "m", "h" for seconds. minutes, hours.
	SyncTimeout Duration
}

Ingest tracks the configuration related to the ingestion protocol.

func NewIngest 

func NewIngest() Ingest

NewIngest returns Ingest with values set to their defaults.

type Logging 

type Logging struct {
	// Level sets the log level for all loggers that do not have a setting in
	// Loggers. The default value is "info".
	Level string
	// Loggers sets log levels for individual loggers.
	Loggers map[string]string
}

Logging configures overall and logger-specific log levels. Level values are case-insensitive and include the following in order of importance: "fatal", "panic", "dpanic", ""error", "warn", "info", "debug"

func NewLogging 

func NewLogging() Logging

NewLogging returns Logging with values set to their defaults.

type Peering 

type Peering struct {
	// Peers lists the nodes to attempt to stay connected with.
	Peers []string
}

Peering configures the peering service. Peering is similar to bootstrapping, but peering maintains connection with all peers configured in the peering service.

func NewPeering 

func NewPeering() Peering

NewPeering returns Peering with values set to their defaults.

func (Peering) PeerAddrs 

func (p Peering) PeerAddrs() ([]peer.AddrInfo, error)

PeerAddrs returns the peering peers as a list of AddrInfo.

type Policy 

type Policy struct {
	// Allow is either false or true, and determines whether a peer is allowed
	// (true) or is blocked (false), by default.
	Allow bool
	// Except is a list of peer IDs that are exceptions to the Allow policy.
	// If Allow is true, then all peers are allowed except those listed in
	// Except. If Allow is false, then no peers are allowed except those listed
	// in Except. in other words, Allow=true means that Except is a deny-list
	// and Allow=false means that Except is an allow-list.
	Except []string

	// Publish determines whether or not peers are allowed to publish
	// advertisements for a provider with a differen peer ID.
	Publish bool
	// PublisherExcept is a list of peer IDs that are exceptions to the Publish
	// policy. If Publish is false, then all allowed peers cannot publish
	// advertisements for providers with a different peer ID, unless listed in
	// PublishExcept. If Publish is true, then all allowed peers can publish
	// advertisements for any provider, unless listed in PublishExcept.
	PublishExcept []string
}

Policy configures which peers are allowed and which may publish on behalf of others. Currently, the allow policy is applied to both providers and publishers. The Publish policie applies only to publishers.

Publishers and providers are not the same. Publishers are peers that supply data to the indexer. Providers are the peers that appear in advertisements and are where clients will retrieve content from.

func NewPolicy 

func NewPolicy() Policy

NewPolicy returns Policy with values set to their defaults.

type Polling 

type Polling struct {
	// ProviderID identifies the provider that this override applies to.
	ProviderID string
	// Interval overrides Discovery.PollInterval.
	Interval Duration
	// RetryAfter overrides Discovery.PollRetryAfter.
	RetryAfter Duration
	// StopAfter overrides Discovery.PollStopAfter.
	StopAfter Duration
}

Polling is a set of polling parameters that is applied to a specific provider. The values override the matching Poll values in the Discovery config.

type RateLimit 

type RateLimit struct {
	// Apply is either false or true, and determines whether a peer is subject
	// to rate limiting (true) or not (false), by default.
	Apply bool
	// Except is a list of peer IDs that are exceptions to the Apply rule. If
	// Apply is false then peers are not rate-limited unless they appear in the
	// Except list. If Apply is true, then only the peers listed in Except are
	// not rate-limited.
	Except []string
	// BlocksPerSecond is the number of blocks allowed to be transferred per
	// second. An advertisement and a block of multihashes are both represented
	// as a block, so this limit applies to both. Setting a value of 0 disables
	// rate limiting, meaning that the rate is infinite.
	BlocksPerSecond int
	// BurstSize is the maximum number of blocks that can be received at once.
	// After this, BlocksPerSecond additional blocks maybe received each
	// second, and any more results in rate limiting. With HTTP ingestion, rate
	// limiting waits until more blocks are allowed to be received. With
	// graphsync, rate limiting terminates the session and resumes it when
	// sufficient time has passed to be able to receive BurstSize blocks. A
	// value of 0 results in 10 times BlocksPerSecond.
	BurstSize int
}

func NewRateLimit 

func NewRateLimit() RateLimit

NewRateLimit returns RateLimit with values set to their defaults.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.