ledgerbackend

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 28, 2021 License: Apache-2.0 Imports: 32 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Base64Ledger 

type Base64Ledger xdr.LedgerCloseMeta

Base64Ledger extends xdr.LedgerCloseMeta with JSON encoding and decoding

func (Base64Ledger) MarshalJSON 

func (r Base64Ledger) MarshalJSON() ([]byte, error)

func (*Base64Ledger) UnmarshalJSON 

func (r *Base64Ledger) UnmarshalJSON(b []byte) error

type CaptiveCoreConfig 

type CaptiveCoreConfig struct {
	// BinaryPath is the file path to the Stellar Core binary
	BinaryPath string
	// ConfigAppendPath is the file path to additional configuration for the Stellar Core configuration file used
	// by captive core. This field is only required when ingesting in online mode (e.g. UnboundedRange).
	ConfigAppendPath string
	// NetworkPassphrase is the Stellar network passphrase used by captive core when connecting to the Stellar network
	NetworkPassphrase string
	// HistoryArchiveURLs are a list of history archive urls
	HistoryArchiveURLs []string

	// CheckpointFrequency is the number of ledgers between checkpoints
	// if unset, DefaultCheckpointFrequency will be used
	CheckpointFrequency uint32
	// LedgerHashStore is an optional store used to obtain hashes for ledger sequences from a trusted source
	LedgerHashStore TrustedLedgerHashStore
	// HTTPPort is the TCP port to listen for requests (defaults to 0, which disables the HTTP server)
	HTTPPort uint
	// Log is an (optional) custom logger which will capture any output from the Stellar Core process.
	// If Log is omitted then all output will be printed to stdout.
	Log *log.Entry
	// Context is the (optional) context which controls the lifetime of a CaptiveStellarCore instance. Once the context is done
	// the CaptiveStellarCore instance will not be able to stream ledgers from Stellar Core or spawn new
	// instances of Stellar Core. If Context is omitted CaptiveStellarCore will default to using context.Background.
	Context context.Context
}

CaptiveCoreConfig contains all the parameters required to create a CaptiveStellarCore instance

type CaptiveStellarCore 

type CaptiveStellarCore struct {
	// contains filtered or unexported fields
}

CaptiveStellarCore is a ledger backend that starts internal Stellar-Core subprocess responsible for streaming ledger data. It provides better decoupling than DatabaseBackend but requires some extra init time.

It operates in two modes:

  • When a BoundedRange is prepared it starts Stellar-Core in catchup mode that replays ledgers in memory. This is very fast but requires Stellar-Core to keep ledger state in RAM. It requires around 3GB of RAM as of August 2020.
  • When a UnboundedRange is prepared it runs Stellar-Core catchup mode to sync with the first ledger and then runs it in a normal mode. This requires the configAppendPath to be provided because a quorum set needs to be selected.

When running CaptiveStellarCore will create a temporary folder to store bucket files and other temporary files. The folder is removed when Close is called.

The communication is performed via filesystem pipe which is created in a temporary folder.

Currently BoundedRange requires a full-trust on history archive. This issue is being fixed in Stellar-Core.

While using BoundedRanges is straightforward there are a few gotchas connected to UnboundedRanges:

  • PrepareRange takes more time because all ledger entries must be stored on disk instead of RAM.
  • If GetLedger is not called frequently (every 5 sec. on average) the Stellar-Core process can go out of sync with the network. This happens because there is no buffering of communication pipe and CaptiveStellarCore has a very small internal buffer and Stellar-Core will not close the new ledger if it's not read.

Except for the Close function, CaptiveStellarCore is not thread-safe and should not be accessed by multiple go routines. Close is thread-safe and can be called from another go routine. Once Close is called it will interrupt and cancel any pending operations.

Requires Stellar-Core v13.2.0+.

func NewCaptive 

func NewCaptive(config CaptiveCoreConfig) (*CaptiveStellarCore, error)

NewCaptive returns a new CaptiveStellarCore instance.

func (*CaptiveStellarCore) Close 

func (c *CaptiveStellarCore) Close() error

Close closes existing Stellar-Core process, streaming sessions and removes all temporary files. Note, once a CaptiveStellarCore instance is closed it can can no longer be used and all subsequent calls to PrepareRange(), GetLedger(), etc will fail. Close is thread-safe and can be called from another go routine.

func (*CaptiveStellarCore) GetLatestLedgerSequence 

func (c *CaptiveStellarCore) GetLatestLedgerSequence() (uint32, error)

GetLatestLedgerSequence returns the sequence of the latest ledger available in the backend. This method returns an error if not in a session (start with PrepareRange).

Note that for UnboundedRange the returned sequence number is not necessarily the latest sequence closed by the network. It's always the last value available in the backend.

func (*CaptiveStellarCore) GetLedger 

func (c *CaptiveStellarCore) GetLedger(sequence uint32) (bool, xdr.LedgerCloseMeta, error)

GetLedger returns true when ledger is found and it's LedgerCloseMeta. Call PrepareRange first to instruct the backend which ledgers to fetch.

CaptiveStellarCore requires PrepareRange call first to initialize Stellar-Core. Requesting a ledger on non-prepared backend will return an error.

Because data is streamed from Stellar-Core ledger after ledger user should request sequences in a non-decreasing order. If the requested sequence number is less than the last requested sequence number, an error will be returned.

This function behaves differently for bounded and unbounded ranges:

  • BoundedRange makes GetLedger blocking if the requested ledger is not yet available in the ledger. After getting the last ledger in a range this method will also Close() the backend.
  • UnboundedRange makes GetLedger non-blocking. The method will return with the first argument equal false.

This is done to provide maximum performance when streaming old ledgers.

func (*CaptiveStellarCore) IsPrepared 

func (c *CaptiveStellarCore) IsPrepared(ledgerRange Range) (bool, error)

IsPrepared returns true if a given ledgerRange is prepared.

func (*CaptiveStellarCore) PrepareRange 

func (c *CaptiveStellarCore) PrepareRange(ledgerRange Range) error

PrepareRange prepares the given range (including from and to) to be loaded. Captive stellar-core backend needs to initalize Stellar-Core state to be able to stream ledgers. Stellar-Core mode depends on the provided ledgerRange:

  • For BoundedRange it will start Stellar-Core in catchup mode.
  • For UnboundedRange it will first catchup to starting ledger and then run it normally (including connecting to the Stellar network).

Please note that using a BoundedRange, currently, requires a full-trust on history archive. This issue is being fixed in Stellar-Core.

type DatabaseBackend 

type DatabaseBackend struct {
	// contains filtered or unexported fields
}

DatabaseBackend implements a database data store.

func NewDatabaseBackend 

func NewDatabaseBackend(dataSourceName, networkPassphrase string) (*DatabaseBackend, error)

func NewDatabaseBackendFromSession 

func NewDatabaseBackendFromSession(session *db.Session, networkPassphrase string) (*DatabaseBackend, error)

func (*DatabaseBackend) Close 

func (dbb *DatabaseBackend) Close() error

Close disconnects an active database session.

func (*DatabaseBackend) GetLatestLedgerSequence 

func (dbb *DatabaseBackend) GetLatestLedgerSequence() (uint32, error)

GetLatestLedgerSequence returns the most recent ledger sequence number present in the database.

func (*DatabaseBackend) GetLedger 

func (dbb *DatabaseBackend) GetLedger(sequence uint32) (bool, xdr.LedgerCloseMeta, error)

GetLedger returns the LedgerCloseMeta for the given ledger sequence number. The first returned value is false when the ledger does not exist in the database.

func (*DatabaseBackend) IsPrepared 

func (*DatabaseBackend) IsPrepared(ledgerRange Range) (bool, error)

IsPrepared returns true if a given ledgerRange is prepared.

func (*DatabaseBackend) PrepareRange 

func (dbb *DatabaseBackend) PrepareRange(ledgerRange Range) error

type HorizonDBLedgerHashStore 

type HorizonDBLedgerHashStore struct {
	// contains filtered or unexported fields
}

HorizonDBLedgerHashStore is a TrustedLedgerHashStore which uses horizon's db to look up ledger hashes

func (HorizonDBLedgerHashStore) GetLedgerHash 

func (h HorizonDBLedgerHashStore) GetLedgerHash(seq uint32) (string, bool, error)

GetLedgerHash returns the ledger hash for the given sequence number

type LatestLedgerSequenceResponse 

type LatestLedgerSequenceResponse struct {
	Sequence uint32 `json:"sequence"`
}

LatestLedgerSequenceResponse is the response for the GetLatestLedgerSequence command.

type LedgerBackend 

type LedgerBackend interface {
	// GetLatestLedgerSequence returns the sequence of the latest ledger available
	// in the backend.
	GetLatestLedgerSequence() (sequence uint32, err error)
	// The first returned value is false when the ledger does not exist in a backend.
	GetLedger(sequence uint32) (bool, xdr.LedgerCloseMeta, error)
	// PrepareRange prepares the given range (including from and to) to be loaded.
	// Some backends (like captive stellar-core) need to initalize data to be
	// able to stream ledgers. Blocks until the first ledger is available.
	PrepareRange(ledgerRange Range) error
	// IsPrepared returns true if a given ledgerRange is prepared.
	IsPrepared(ledgerRange Range) (bool, error)
	Close() error
}

LedgerBackend represents the interface to a ledger data store.

type LedgerResponse 

type LedgerResponse struct {
	Present bool         `json:"present"`
	Ledger  Base64Ledger `json:"ledger"`
}

LedgerResponse is the response for the GetLedger command.

type MockDatabaseBackend 

type MockDatabaseBackend struct {
	mock.Mock
}

func (*MockDatabaseBackend) Close 

func (m *MockDatabaseBackend) Close() error

func (*MockDatabaseBackend) GetLatestLedgerSequence 

func (m *MockDatabaseBackend) GetLatestLedgerSequence() (uint32, error)

func (*MockDatabaseBackend) GetLedger 

func (m *MockDatabaseBackend) GetLedger(sequence uint32) (bool, xdr.LedgerCloseMeta, error)

func (*MockDatabaseBackend) IsPrepared 

func (m *MockDatabaseBackend) IsPrepared(ledgerRange Range) (bool, error)

func (*MockDatabaseBackend) PrepareRange 

func (m *MockDatabaseBackend) PrepareRange(ledgerRange Range) error

type MockLedgerHashStore 

type MockLedgerHashStore struct {
	mock.Mock
}

MockLedgerHashStore is a mock implementation of TrustedLedgerHashStore

func (*MockLedgerHashStore) GetLedgerHash 

func (m *MockLedgerHashStore) GetLedgerHash(seq uint32) (string, bool, error)

GetLedgerHash returns the ledger hash for the given sequence number

type PrepareRangeResponse 

type PrepareRangeResponse struct {
	LedgerRange   Range     `json:"ledgerRange"`
	StartTime     time.Time `json:"startTime"`
	Ready         bool      `json:"ready"`
	ReadyDuration int       `json:"readyDuration"`
}

PrepareRangeResponse describes the status of the pending PrepareRange operation.

type Range 

type Range struct {
	// contains filtered or unexported fields
}

Range represents a range of ledger sequence numbers.

func BoundedRange 

func BoundedRange(from uint32, to uint32) Range

BoundedRange constructs a bounded range of ledgers with a fixed starting ledger and ending ledger.

func SingleLedgerRange 

func SingleLedgerRange(ledger uint32) Range

SingleLedgerRange constructs a bounded range containing a single ledger.

func UnboundedRange 

func UnboundedRange(from uint32) Range

BoundedRange constructs a unbounded range of ledgers with a fixed starting ledger.

func (Range) Contains 

func (r Range) Contains(other Range) bool

func (Range) MarshalJSON 

func (r Range) MarshalJSON() ([]byte, error)

func (Range) String 

func (r Range) String() string

func (*Range) UnmarshalJSON 

func (r *Range) UnmarshalJSON(b []byte) error

type RemoteCaptiveOption 

type RemoteCaptiveOption func(c *RemoteCaptiveStellarCore)

RemoteCaptiveOption values can be passed into NewRemoteCaptive to customize a RemoteCaptiveStellarCore instance.

func PrepareRangePollInterval 

func PrepareRangePollInterval(d time.Duration) RemoteCaptiveOption

PrepareRangePollInterval configures how often the captive core server will be polled when blocking on the PrepareRange operation.

type RemoteCaptiveStellarCore 

type RemoteCaptiveStellarCore struct {
	// contains filtered or unexported fields
}

RemoteCaptiveStellarCore is an http client for interacting with a remote captive core server.

func NewRemoteCaptive 

func NewRemoteCaptive(captiveCoreURL string, options ...RemoteCaptiveOption) (RemoteCaptiveStellarCore, error)

NewRemoteCaptive returns a new RemoteCaptiveStellarCore instance.

Only the captiveCoreURL parameter is required.

func (RemoteCaptiveStellarCore) Close 

func (c RemoteCaptiveStellarCore) Close() error

Close cancels any pending PrepareRange requests.

func (RemoteCaptiveStellarCore) GetLatestLedgerSequence 

func (c RemoteCaptiveStellarCore) GetLatestLedgerSequence() (sequence uint32, err error)

GetLatestLedgerSequence returns the sequence of the latest ledger available in the backend. This method returns an error if not in a session (start with PrepareRange).

Note that for UnboundedRange the returned sequence number is not necessarily the latest sequence closed by the network. It's always the last value available in the backend.

func (RemoteCaptiveStellarCore) GetLedger 

func (c RemoteCaptiveStellarCore) GetLedger(sequence uint32) (bool, xdr.LedgerCloseMeta, error)

GetLedger returns true when ledger is found and it's LedgerCloseMeta. Call PrepareRange first to instruct the backend which ledgers to fetch.

CaptiveStellarCore requires PrepareRange call first to initialize Stellar-Core. Requesting a ledger on non-prepared backend will return an error.

Because data is streamed from Stellar-Core ledger after ledger user should request sequences in a non-decreasing order. If the requested sequence number is less than the last requested sequence number, an error will be returned.

This function behaves differently for bounded and unbounded ranges:

  • BoundedRange makes GetLedger blocking if the requested ledger is not yet available in the ledger. After getting the last ledger in a range this method will also Close() the backend.
  • UnboundedRange makes GetLedger non-blocking. The method will return with the first argument equal false.

This is done to provide maximum performance when streaming old ledgers.

func (RemoteCaptiveStellarCore) IsPrepared 

func (c RemoteCaptiveStellarCore) IsPrepared(ledgerRange Range) (bool, error)

IsPrepared returns true if a given ledgerRange is prepared.

func (RemoteCaptiveStellarCore) PrepareRange 

func (c RemoteCaptiveStellarCore) PrepareRange(ledgerRange Range) error

PrepareRange prepares the given range (including from and to) to be loaded. Captive stellar-core backend needs to initalize Stellar-Core state to be able to stream ledgers. Stellar-Core mode depends on the provided ledgerRange:

  • For BoundedRange it will start Stellar-Core in catchup mode.
  • For UnboundedRange it will first catchup to starting ledger and then run it normally (including connecting to the Stellar network).

Please note that using a BoundedRange, currently, requires a full-trust on history archive. This issue is being fixed in Stellar-Core.

type TrustedLedgerHashStore 

type TrustedLedgerHashStore interface {
	// GetLedgerHash returns the ledger hash for the given sequence number
	GetLedgerHash(seq uint32) (string, bool, error)
}

TrustedLedgerHashStore is used to query ledger data from a trusted source. The store should contain ledgers verified by Stellar-Core, do not use untrusted source like history archives.

func NewHorizonDBLedgerHashStore 

func NewHorizonDBLedgerHashStore(session *db.Session) TrustedLedgerHashStore

NewHorizonDBLedgerHashStore constructs a new TrustedLedgerHashStore backed by the horizon db

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.