scraper

Documentation

Index

• func NormalizeTradeAssets(t *hProtocol.Trade)
• type FinalAsset
• type OrderbookStats
• type ScraperConfig
  • func (c *ScraperConfig) FetchAllTrades(since time.Time, limit int) (trades []hProtocol.Trade, err error)
  • func (c *ScraperConfig) FetchOrderbookForAssets(bType, bCode, bIssuer, cType, cCode, cIssuer string) (OrderbookStats, error)
  • func (c *ScraperConfig) ProcessAllAssets(limit int, parallelism int, assetQueue chan<- FinalAsset) (numNonTrash int, numTrash int)
  • func (c *ScraperConfig) StreamNewTrades(cursor string, h auroraclient.TradeHandler) error
• type TOMLCache
  • func (c TOMLCache) Get(tomlURL string) (tomlIssuer TOMLIssuer, ok bool)
  • func (c *TOMLCache) Set(tomlURL string, tomlIssuer TOMLIssuer)
• type TOMLCurrency
• type TOMLDoc
• type TOMLIssuer

Functions

func NormalizeTradeAssets

func NormalizeTradeAssets(t *hProtocol.Trade)

NormalizeTradeAssets enforces the following rules: 1. native asset type refers to a "XLM" code and a "native" issuer 2. native is always the base asset (and if not, base and counter are swapped) 3. when trades are between two non-native, the base is the asset whose string comes first alphabetically.

Types

type FinalAsset

type FinalAsset struct {
	Code                        string     `json:"code"`
	Issuer                      string     `json:"issuer"`
	Type                        string     `json:"type"`
	NumAccounts                 int32      `json:"num_accounts"`
	AuthRequired                bool       `json:"auth_required"`
	AuthRevocable               bool       `json:"auth_revocable"`
	Amount                      float64    `json:"amount"`
	IssuerDetails               TOMLIssuer `json:"-"`
	AssetControlledByDomain     bool       `json:"asset_controlled_by_domain"`
	Error                       string     `json:"-"`
	AnchorAsset                 string     `json:"anchor_asset"`
	AnchorAssetType             string     `json:"anchor_asset_type"`
	IsValid                     bool       `json:"-"`
	LastValid                   time.Time  `json:"-"`
	LastChecked                 time.Time  `json:"-"`
	IsTrash                     bool       `json:"-"`
	DisplayDecimals             int        `json:"display_decimals"`
	Name                        string     `json:"name"`
	Desc                        string     `json:"desc"`
	Conditions                  string     `json:"conditions"`
	IsAssetAnchored             bool       `json:"is_asset_anchored"`
	FixedNumber                 int        `json:"fixed_number"`
	MaxNumber                   int        `json:"max_number"`
	IsUnlimited                 bool       `json:"is_unlimited"`
	RedemptionInstructions      string     `json:"redemption_instructions"`
	CollateralAddresses         []string   `json:"collateral_addresses"`
	CollateralAddressSignatures []string   `json:"collateral_address_signatures"`
	Countries                   string     `json:"countries"`
	Status                      string     `json:"status"`
}

FinalAsset is the interface to represent the aggregated Asset data.

type OrderbookStats

type OrderbookStats struct {
	BaseAssetCode      string
	BaseAssetType      string
	BaseAssetIssuer    string
	CounterAssetCode   string
	CounterAssetType   string
	CounterAssetIssuer string
	NumBids            int
	BidVolume          float64
	HighestBid         float64
	NumAsks            int
	AskVolume          float64
	LowestAsk          float64
	Spread             float64
	SpreadMidPoint     float64
}

OrderbookStats represents the Orderbook stats for a given asset

type ScraperConfig

type ScraperConfig struct {
	Client auroraclient.ClientInterface
	Logger *hlog.Entry
	Ctx    *context.Context
}

func (*ScraperConfig) FetchAllTrades

func (c *ScraperConfig) FetchAllTrades(since time.Time, limit int) (trades []hProtocol.Trade, err error)

FetchAllTrades fetches all trades for a given period, respecting the limit. If limit = 0, will fetch all trades for that given period.

func (*ScraperConfig) FetchOrderbookForAssets

func (c *ScraperConfig) FetchOrderbookForAssets(bType, bCode, bIssuer, cType, cCode, cIssuer string) (OrderbookStats, error)

FetchOrderbookForAssets fetches the orderbook stats for the base and counter assets provided in the parameters

func (*ScraperConfig) ProcessAllAssets

func (c *ScraperConfig) ProcessAllAssets(limit int, parallelism int, assetQueue chan<- FinalAsset) (numNonTrash int, numTrash int)

ProcessAllAssets fetches assets from the Aurora public net. If limit = 0, will fetch all assets.

func (*ScraperConfig) StreamNewTrades

func (c *ScraperConfig) StreamNewTrades(cursor string, h auroraclient.TradeHandler) error

StreamNewTrades streams trades directly from aurora and calls the handler function whenever a new trade appears.

type TOMLCache

type TOMLCache struct {
	// contains filtered or unexported fields
}

TOMLCache caches a max of one toml at a time. Old records are discarded when new records are added.

func (TOMLCache) Get

func (c TOMLCache) Get(tomlURL string) (tomlIssuer TOMLIssuer, ok bool)

Get returns the toml for the given URL if it is cached. Ok is true only if the toml returned was found in the cache.

func (*TOMLCache) Set

func (c *TOMLCache) Set(tomlURL string, tomlIssuer TOMLIssuer)

Set adds the given toml to the cache using the toml URL as its key for lookup later.

type TOMLCurrency

type TOMLCurrency struct {
	Code                        string   `toml:"code"`
	Issuer                      string   `toml:"issuer"`
	IsAssetAnchored             bool     `toml:"is_asset_anchored"`
	AnchorAsset                 string   `toml:"anchor_asset"`
	AnchorAssetType             string   `toml:"anchor_asset_type"`
	DisplayDecimals             int      `toml:"display_decimals"`
	Name                        string   `toml:"name"`
	Desc                        string   `toml:"desc"`
	Conditions                  string   `toml:"conditions"`
	FixedNumber                 int      `toml:"fixed_number"`
	MaxNumber                   int      `toml:"max_number"`
	IsUnlimited                 bool     `toml:"is_unlimited"`
	RedemptionInstructions      string   `toml:"redemption_instructions"`
	CollateralAddresses         []string `toml:"collateral_addresses"`
	CollateralAddressSignatures []string `toml:"collateral_address_signatures"`
	Status                      string   `toml:"status"`
}

TOMLCurrency is the interface for storing TOML Currency Information. See: https://github.com/HashCash-Consultants/hcnet-protocol/blob/master/ecosystem/sep-0001.md#currency-documentation

type TOMLDoc

type TOMLDoc struct {
	OrgName    string `toml:"ORG_NAME"`
	OrgURL     string `toml:"ORG_URL"`
	OrgTwitter string `toml:"ORG_TWITTER"`
}

TOMLDoc is the interface for storing TOML Issuer Documentation. See: https://github.com/HashCash-Consultants/hcnet-protocol/blob/master/ecosystem/sep-0001.md#currency-documentation

type TOMLIssuer

type TOMLIssuer struct {
	FederationServer string         `toml:"FEDERATION_SERVER"`
	AuthServer       string         `toml:"AUTH_SERVER"`
	TransferServer   string         `toml:"TRANSFER_SERVER"`
	WebAuthEndpoint  string         `toml:"WEB_AUTH_ENDPOINT"`
	SigningKey       string         `toml:"SIGNING_KEY"`
	DepositServer    string         `toml:"DEPOSIT_SERVER"` // for legacy purposes
	Documentation    TOMLDoc        `toml:"DOCUMENTATION"`
	Currencies       []TOMLCurrency `toml:"CURRENCIES"`
	TOMLURL          string         `toml:"-"`
}

TOMLIssuer is the interface for storing TOML Issuer Information. See: https://github.com/HashCash-Consultants/hcnet-protocol/blob/master/ecosystem/sep-0001.md#currency-documentation