data

package
v0.1.28 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 10, 2025 License: AGPL-3.0 Imports: 29 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ConcurNum = 5 // 并发处理的数量

)

Functions

func Build1mWithTicks 

func Build1mWithTicks(args *config.CmdArgs) *errs.Error

func CalcFilePerfs 

func CalcFilePerfs(args *config.CmdArgs) *errs.Error

CalcFilePerfs calc sharpe/sortino ratio for input data

func FindPathNames 

func FindPathNames(inPath, suffix string) ([]string, *errs.Error)

FindPathNames Finds all file paths of the specified type for a given path

An array of paths is returned, with the first being the parent directory followed by the relative child paths 查找给定路径所有指定类型的文件路径

返回路径数组，第一个是父目录，后续是相对子路径

func ReadZipCSVs 

func ReadZipCSVs(inPath string, pBar *utils.PrgBar, handle FuncReadZipItem, arg interface{}) *errs.Error

func RunFormatTick 

func RunFormatTick(args *config.CmdArgs) *errs.Error

func RunHistFeeders 

func RunHistFeeders(makeFeeders func() []IHistKlineFeeder, versions chan int, pBar *utils.PrgBar) *errs.Error

RunHistFeeders run hist feeders for historical data

versions: When an integer greater than the previous value is received, makeFeeders will be called to re-acquire and continue running; when a negative number is received, exit immediately

pBar: optional, used to display a progress bar

func RunSpider 

func RunSpider(addr string) *errs.Error

Types

type DBKlineFeeder 

type DBKlineFeeder struct {
	HistKLineFeeder
	// contains filtered or unexported fields
}

DBKlineFeeder The database reads the K-line Feeder for backtesting 数据库读取K线的Feeder，用于回测

func NewDBKlineFeeder 

func NewDBKlineFeeder(exs *orm.ExSymbol, callBack FnPairKline, showLog bool) (*DBKlineFeeder, *errs.Error)

func (*DBKlineFeeder) DownIfNeed 

func (f *DBKlineFeeder) DownIfNeed(sess *orm.Queries, exchange banexg.BanExchange, pBar *utils.PrgBar) *errs.Error

DownIfNeed Download data for a specified interval pBar is used for progress update, the total is 1000, and the amount is updated each time 下载指定区间的数据 pBar 用于进度更新，总和为1000，每次更新此次的量

func (*DBKlineFeeder) SetSeek 

func (f *DBKlineFeeder) SetSeek(since int64)

type Feeder 

type Feeder struct {
	*orm.ExSymbol
	States   []*PairTFCache
	WaitBar  *banexg.Kline
	CallBack FnPairKline
	OnEnvEnd FuncEnvEnd // If the futures main force switches or the stock is ex-rights, the position needs to be closed first 期货主力切换或股票除权，需先平仓
	// contains filtered or unexported fields
}

Feeder Each Feeder corresponds to a trading pair. Can contain multiple time dimensions.

Supports dynamic addition of time dimension. Backtest mode: Call execution callbacks in sequence according to the next update time of the Feeder. Real mode: Subscribe to new data for this trading pair's time period and execute a callback when it is awakened. Support warm-up data. Each strategy + trading pair is preheated independently throughout the entire process, and cross-preheating is not allowed to avoid btime contamination. LiveFeeder requires preheating for both new trading pairs and new cycles; HistFeeder only requires preheating for new cycles. 每个Feeder对应一个交易对。可包含多个时间维度。 

支持动态添加时间维度。
回测模式：根据Feeder的下次更新时间，按顺序调用执行回调。
实盘模式：订阅此交易对时间周期的新数据，被唤起时执行回调。
支持预热数据。每个策略+交易对全程单独预热，不可交叉预热，避免btime被污染。
LiveFeeder新交易对和新周期都需要预热；HistFeeder仅新周期需要预热

func (*Feeder) SubTfs 

func (f *Feeder) SubTfs(timeFrames []string, delOther bool) []string

subTfs Add monitoring to States and return the newly added TimeFrames 添加监听到States中，返回新增的TimeFrames

type FetchJob 

type FetchJob struct {
	PairTFCache
	Pair      string
	CheckSecs int
	Since     int64
	NextRun   int64
}

type FnPairKline 

type FnPairKline = func(bar *orm.InfoKline)

type FuncConvert 

type FuncConvert func(inPath string, file *zip.File, writer *zip.Writer) *errs.Error

type FuncEnvEnd 

type FuncEnvEnd = func(bar *banexg.PairTFKline, adj *orm.AdjInfo)

type FuncReadZipItem 

type FuncReadZipItem func(inPath string, fid int, file *zip.File, arg interface{}) *errs.Error

type FuncTickBar 

type FuncTickBar func(inPath string, row []string) (string, int64, [5]float64)

type HistKLineFeeder 

type HistKLineFeeder struct {
	KlineFeeder
	TimeRange *config.TimeTuple

	TradeTimes [][2]int64 // Trading time 可交易时间
	// contains filtered or unexported fields
}

HistKLineFeeder Historical data feedback device. Is the base class for file feedback and database feedback.

Backtest mode: Read 3K bars each time, and backtest triggers in sequence according to nextMS size. 历史数据反馈器。是文件反馈器和数据库反馈器的基类。 

回测模式：每次读取3K个bar，按nextMS大小依次回测触发。

func (*HistKLineFeeder) CallNext 

func (f *HistKLineFeeder) CallNext()

func (*HistKLineFeeder) GetBar 

func (f *HistKLineFeeder) GetBar() *banexg.Kline

Get the current bar for invokeBar; callNext should be called afterwards to set the cursor to the next bar. 获取当前bar，用于invokeBar；之后应调用callNext设置光标到下一个bar

func (*HistKLineFeeder) RunBar 

func (f *HistKLineFeeder) RunBar(bar *banexg.Kline) *errs.Error

func (*HistKLineFeeder) WarmTfs added in v0.1.12 

func (f *HistKLineFeeder) WarmTfs(curMS int64, tfNums map[string]int, pBar *utils.PrgBar) (int64, *errs.Error)

type HistProvider 

type HistProvider struct {
	Provider[IHistKlineFeeder]
	// contains filtered or unexported fields
}

func NewHistProvider 

func NewHistProvider(callBack FnPairKline, envEnd FuncEnvEnd, showLog bool, pBar *utils.StagedPrg) *HistProvider

func (*HistProvider) LoopMain 

func (p *HistProvider) LoopMain() *errs.Error

func (*HistProvider) SubWarmPairs 

func (p *HistProvider) SubWarmPairs(items map[string]map[string]int, delOther bool) *errs.Error

func (*HistProvider) Terminate 

func (p *HistProvider) Terminate()

func (*HistProvider) UnSubPairs 

func (p *HistProvider) UnSubPairs(pairs ...string) *errs.Error

type IHistKlineFeeder 

type IHistKlineFeeder interface {
	IKlineFeeder

	/*
		DownIfNeed Download the entire range of K lines, which needs to be called before SetSeek  下载整个范围的K线，需在SetSeek前调用
	*/
	DownIfNeed(sess *orm.Queries, exchange banexg.BanExchange, pBar *utils.PrgBar) *errs.Error
	/*
		SetSeek Set the reading position and call it before loop reading   设置读取位置，在循环读取前调用
	*/
	SetSeek(since int64)
	/*
		GetBar Get the current K line, and then call CallNext to move the pointer to the next 获取当前K线，然后可调用CallNext移动指针到下一个
	*/
	GetBar() *banexg.Kline
	/*
		RunBar Run the callback function corresponding to Bar 运行Bar对应的回调函数
	*/
	RunBar(bar *banexg.Kline) *errs.Error
	/*
		CallNext Move the pointer to the next K line 移动指针到下一个K线
	*/
	CallNext()
	// contains filtered or unexported methods
}

func SortFeeders 

func SortFeeders(holds []IHistKlineFeeder, hold IHistKlineFeeder, insert bool) []IHistKlineFeeder

type IKlineFeeder 

type IKlineFeeder interface {

	/*
		SubTfs Subscribe to data for a specified time period for the current target. Multiple 为当前标的订阅指定时间周期的数据，可多个
	*/
	SubTfs(timeFrames []string, delOther bool) []string
	/*
		WarmTfs The preheating time period gives the number of K lines to the specified time. 预热时间周期给定K线数量到指定时间
	*/
	WarmTfs(curMS int64, tfNums map[string]int, pBar *utils.PrgBar) (int64, *errs.Error)
	// contains filtered or unexported methods
}

type IProvider 

type IProvider interface {
	LoopMain() *errs.Error
	SubWarmPairs(items map[string]map[string]int, delOther bool) *errs.Error
	UnSubPairs(pairs ...string) *errs.Error
	SetDirty()
}

type KLineMsg 

type KLineMsg struct {
	NotifyKLines
	ExgName string // The name of the exchange 交易所名称
	Market  string // market 市场
	Pair    string // symbol  币种
}

type KLineWatcher 

type KLineWatcher struct {
	*utils.ClientIO

	OnKLineMsg func(msg *KLineMsg) // 收到爬虫K线消息
	OnTrade    func(exgName, market string, trade *banexg.Trade)
	// contains filtered or unexported fields
}

func NewKlineWatcher 

func NewKlineWatcher(addr string) (*KLineWatcher, *errs.Error)

func (*KLineWatcher) SendMsg 

func (w *KLineWatcher) SendMsg(action string, data interface{}) *errs.Error

func (*KLineWatcher) UnWatchJobs 

func (w *KLineWatcher) UnWatchJobs(exgName, marketType, jobType string, pairs []string) *errs.Error

func (*KLineWatcher) WatchJobs 

func (w *KLineWatcher) WatchJobs(exgName, marketType, jobType string, jobs ...WatchJob) *errs.Error

WatchJobs Subscribe data from crawlers. 从爬虫订阅数据。ohlcv/uohlcv/ws/trade/book

type KlineFeeder 

type KlineFeeder struct {
	Feeder
	PreFire float64 // Ratio of triggering bar early 提前触发bar的比率
	// contains filtered or unexported fields
}

KlineFeeder Each Feeder corresponds to a trading pair. Can contain multiple time dimensions. Real use.

Supports dynamic addition of time dimension. Supports returning preheating data. Each strategy + trading pair is preheated independently throughout the entire process, and cross-preheating is not allowed to avoid btime contamination.

Backtest mode: Use derived structure: DbKlineFeeder

Real mode: Subscribe to new data for this trading pair's time period and execute a callback when it is awakened. Check whether this trading pair has been refreshed in the spider monitor. If not, send a message to the crawler monitor. 每个Feeder对应一个交易对。可包含多个时间维度。实盘使用。 

支持动态添加时间维度。
支持返回预热数据。每个策略+交易对全程单独预热，不可交叉预热，避免btime被污染。

回测模式：使用派生结构体：DbKlineFeeder

实盘模式：订阅此交易对时间周期的新数据，被唤起时执行回调。
检查此交易对是否已在spider监听刷新，如没有则发消息给爬虫监听。

func NewKlineFeeder 

func NewKlineFeeder(exs *orm.ExSymbol, callBack FnPairKline, showLog bool) (*KlineFeeder, *errs.Error)

func (*KlineFeeder) WarmTfs 

func (f *KlineFeeder) WarmTfs(curMS int64, tfNums map[string]int, pBar *utils.PrgBar) (int64, *errs.Error)

type LiveProvider 

type LiveProvider struct {
	Provider[IKlineFeeder]
	*KLineWatcher
}

func NewLiveProvider 

func NewLiveProvider(callBack FnPairKline, envEnd FuncEnvEnd) (*LiveProvider, *errs.Error)

func (*LiveProvider) LoopMain 

func (p *LiveProvider) LoopMain() *errs.Error

func (*LiveProvider) SubWarmPairs 

func (p *LiveProvider) SubWarmPairs(items map[string]map[string]int, delOther bool) *errs.Error

func (*LiveProvider) UnSubPairs 

func (p *LiveProvider) UnSubPairs(pairs ...string) *errs.Error

type LiveSpider 

type LiveSpider struct {
	*utils.ServerIO
	// contains filtered or unexported fields
}
var (
	Spider *LiveSpider
)

type Miner 

type Miner struct {
	ExgName string
	Market  string

	Fetchs       map[string]*FetchJob
	KlineReady   bool
	KlinePairs   map[string]bool
	TradeReady   bool
	TradePairs   map[string]bool
	BookReady    bool
	BookPairs    map[string]bool
	IsWatchPrice bool
	// contains filtered or unexported fields
}

func (*Miner) SubPairs 

func (m *Miner) SubPairs(jobType string, pairs ...string) *errs.Error

func (*Miner) UnSubPairs 

func (m *Miner) UnSubPairs(jobType string, pairs ...string) *errs.Error

type NotifyKLines 

type NotifyKLines struct {
	TFSecs   int
	Interval int // 推送更新间隔, <= TFSecs
	Arr      []*banexg.Kline
}

type PairTFCache 

type PairTFCache struct {
	TimeFrame  string
	TFSecs     int
	NextMS     int64         // Record the start timestamp of the next bar expected to be received. If it is inconsistent, the bar is missing and needs to be queried and updated. 记录下一个期待收到的bar起始时间戳，如果不一致，则出现了bar缺失，需查询更新。
	WaitBar    *banexg.Kline // Record unfinished bars. Should be set to nil when completed 记录尚未完成的bar。已完成时应置为nil
	Latest     *banexg.Kline // Record the latest bar data, which may not be completed or may be completed 记录最新bar数据，可能未完成，可能已完成
	AlignOffMS int64
}

type Provider 

type Provider[T IKlineFeeder] struct {
	// contains filtered or unexported fields
}

func (*Provider[IKlineFeeder]) SetDirty 

func (p *Provider[IKlineFeeder]) SetDirty()

func (*Provider[IKlineFeeder]) SubWarmPairs 

func (p *Provider[IKlineFeeder]) SubWarmPairs(items map[string]map[string]int, delOther bool, pBar *utils.StagedPrg) ([]IKlineFeeder, map[string]int64, []string, *errs.Error)

SubWarmPairs Add new trading pair subscription from data provider.

items: pair[timeFrame]warmNum Return the trading pairs with the smallest period change (new/old pairs new period), warm-up tasks 从数据提供者添加新的交易对订阅。 

items: pair[timeFrame]warmNum
返回最小周期变化的交易对(新增/旧对新周期)、预热任务

func (*Provider[IKlineFeeder]) UnSubPairs 

func (p *Provider[IKlineFeeder]) UnSubPairs(pairs ...string) []string

type SaveKline 

type SaveKline struct {
	Sid       int32
	TimeFrame string
	Arr       []*banexg.Kline
	SkipFirst bool
	MsgAction string
}

type SubKLineState 

type SubKLineState struct {
	Sid        int32
	NextNotify float64
	PrevBar    *banexg.Kline
}

type WarmJob 

type WarmJob struct {
	// contains filtered or unexported fields
}

type WatchJob 

type WatchJob struct {
	Symbol    string
	TimeFrame string
	Since     int64
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.