orm

package
v0.1.22 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 2, 2025 License: AGPL-3.0 Imports: 30 Imported by: 2

README

数据库概述

需要存储到数据库的有三类：K线数据、交易数据、UI相关数据。
为确保性能，K线使用TimeScaledb存储。
为确保灵活性，交易数据(ormo)和UI相关数据(ormu)使用独立的sqlite文件存储。
ormo/ormu依赖orm，不可反向依赖，避免出现依赖环

sqlc

数据库相关的访问代码全部使用sqlc从sql文件生成go代码。
kline1m等不需要sqlc生成的，全部写到schema2.sql中。
sqlc不支持windows直接使用，需要启动Docker Desktop，然后在命令行执行下面命令：

docker run --rm -v "E:/trade/go/banbot/orm:/src" -w /src sqlc/sqlc generate

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	DbTrades = "trades"
	DbUI     = "ui"
)

Functions

func ApplyAdj 

func ApplyAdj(adjs []*AdjInfo, klines []*banexg.Kline, adj int, cutEnd int64, limit int) []*banexg.Kline

ApplyAdj Calculate the K-line after adjustment 计算复权后K线 adjs Must be in ascending order 必须已升序 cutEnd Maximum end time of interception 截取的最大结束时间 adj Type of adjustment of Rights 复权类型 limit 返回数量

func BulkDownOHLCV 

func BulkDownOHLCV(exchange banexg.BanExchange, exsList map[int32]*ExSymbol, timeFrame string, startMS, endMS int64, limit int, prg utils.PrgCB) *errs.Error

BulkDownOHLCV Batch simultaneous download of K-line 批量同时下载K线

func CalcAdjFactors 

func CalcAdjFactors(args *config.CmdArgs) *errs.Error

CalcAdjFactors Calculate and update all weighting factors 计算更新所有复权因子

func DbLite added in v0.1.12 

func DbLite(src string, path string, write bool) (*sql.DB, *errs.Error)

func EnsureCurSymbols 

func EnsureCurSymbols(symbols []string) *errs.Error

func EnsureExgSymbols 

func EnsureExgSymbols(exchange banexg.BanExchange) *errs.Error

func EnsureListDates added in v0.1.21 

func EnsureListDates(sess *Queries, exchange banexg.BanExchange, exsMap map[int32]*ExSymbol, exsList []*ExSymbol) *errs.Error

func EnsureSymbols 

func EnsureSymbols(symbols []*ExSymbol, exchanges ...string) *errs.Error

func FastBulkOHLCV 

func FastBulkOHLCV(exchange banexg.BanExchange, symbols []string, timeFrame string,
	startMS, endMS int64, limit int, handler func(string, string, []*banexg.Kline, []*AdjInfo)) *errs.Error

FastBulkOHLCV Quickly obtain K-lines in bulk. Download all the required currencies first, then perform batch queries and group returns. Suitable for situations where there are multiple currencies, the required start and end times are consistent, and most of them have already been downloaded. For combination varieties, return the unweighted candlestick and the weighting factor, and call ApplyAdj for weighting as needed 快速批量获取K线。先下载所有需要的币种，然后批量查询再分组返回。 

适用于币种较多，且需要的开始结束时间一致，且大部分已下载的情况。
对于组合品种，返回未复权的K线，和复权因子，自行根据需要调用ApplyAdj复权

func FetchApiOHLCV 

func FetchApiOHLCV(ctx context.Context, exchange banexg.BanExchange, pair, timeFrame string, startMS, endMS int64, out chan []*banexg.Kline) *errs.Error

FetchApiOHLCV Download the K-line data of the trading pair according to the given time period. If you need to download from the end to the beginning, you should make startMS>endMS 按给定时间段下载交易对的K线数据。 如果需要从后往前下载，应该使startMS>endMS

func GetAlignOff 

func GetAlignOff(sid int32, toTfMSecs int64) int64

func GetDownTF 

func GetDownTF(timeFrame string) (string, *errs.Error)

GetDownTF Retrieve the download time period corresponding to the specified period. Only 1m and 1h allow downloading and writing to the super table. All other dimensions are aggregated from these two dimensions. 

获取指定周期对应的下载的时间周期。
只有1m和1h允许下载并写入超表。其他维度都是由这两个维度聚合得到。

func GetExSymbols 

func GetExSymbols(exgName, market string) map[int32]*ExSymbol

func InitExg 

func InitExg(exchange banexg.BanExchange) *errs.Error

func InitListDates 

func InitListDates() *errs.Error

func LoadAllExSymbols 

func LoadAllExSymbols() *errs.Error

func LoadMarkets 

func LoadMarkets(exchange banexg.BanExchange, reload bool) (banexg.MarketMap, *errs.Error)

func MapExSymbols 

func MapExSymbols(exchange banexg.BanExchange, symbols []string) (map[int32]*ExSymbol, *errs.Error)

func NewDbErr 

func NewDbErr(code int, err_ error) *errs.Error

func SetDbPath added in v0.1.12 

func SetDbPath(key, path string)

func Setup 

func Setup() *errs.Error

func SyncKlineTFs 

func SyncKlineTFs(args *config.CmdArgs, pb *utils.StagedPrg) *errs.Error

SyncKlineTFs Check the data consistency of each kline table. If there is more low dimensional data than high dimensional data, aggregate and update to high dimensional data 检查各kline表的数据一致性，如果低维度数据比高维度多，则聚合更新到高维度

Types

type AddAdjFactorsParams 

type AddAdjFactorsParams struct {
	Sid     int32   `json:"sid"`
	SubID   int32   `json:"sub_id"`
	StartMs int64   `json:"start_ms"`
	Factor  float64 `json:"factor"`
}

type AddCalendarsParams 

type AddCalendarsParams struct {
	Name    string `json:"name"`
	StartMs int64  `json:"start_ms"`
	StopMs  int64  `json:"stop_ms"`
}

type AddInsKlineParams added in v0.1.5 

type AddInsKlineParams struct {
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
	StartMs   int64  `json:"start_ms"`
	StopMs    int64  `json:"stop_ms"`
}

type AddKHolesParams 

type AddKHolesParams struct {
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
	Start     int64  `json:"start"`
	Stop      int64  `json:"stop"`
}

type AddKInfoParams 

type AddKInfoParams struct {
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
	Start     int64  `json:"start"`
	Stop      int64  `json:"stop"`
}

type AddSymbolsParams 

type AddSymbolsParams struct {
	Exchange string `json:"exchange"`
	ExgReal  string `json:"exg_real"`
	Market   string `json:"market"`
	Symbol   string `json:"symbol"`
}

type AdjFactor 

type AdjFactor struct {
	ID      int32   `json:"id"`
	Sid     int32   `json:"sid"`
	SubID   int32   `json:"sub_id"`
	StartMs int64   `json:"start_ms"`
	Factor  float64 `json:"factor"`
}

type AdjInfo 

type AdjInfo struct {
	*ExSymbol
	Factor    float64 // Original adjacent weighting factor 原始相邻复权因子
	CumFactor float64 // Cumulative weighting factor 累计复权因子
	StartMS   int64   // start timestamp 开始时间
	StopMS    int64   // stop timestamp 结束时间
}

func AutoFetchOHLCV 

func AutoFetchOHLCV(exchange banexg.BanExchange, exs *ExSymbol, timeFrame string, startMS, endMS int64,
	limit int, withUnFinish bool, pBar *utils.PrgBar) ([]*AdjInfo, []*banexg.Kline, *errs.Error)

AutoFetchOHLCV 

Get K-line data for a given trading pair, a given time dimension, and a given range.
Try to read from local first, download from the exchange if it doesn't exist, and then return.
获取给定交易对，给定时间维度，给定范围的K线数据。
先尝试从本地读取，不存在时从交易所下载，然后返回。

func GetOHLCV 

func GetOHLCV(exs *ExSymbol, timeFrame string, startMS, endMS int64, limit int, withUnFinish bool) ([]*AdjInfo, []*banexg.Kline, *errs.Error)

GetOHLCV Get the variety K-line, if you need to rebalance, it will be automatically reweighted 获取品种K线，如需复权自动前复权

func (*AdjInfo) Apply 

func (a *AdjInfo) Apply(bars []*banexg.Kline, adj int) []*banexg.Kline

type Calendar 

type Calendar struct {
	ID      int32  `json:"id"`
	Name    string `json:"name"`
	StartMs int64  `json:"start_ms"`
	StopMs  int64  `json:"stop_ms"`
}

type DBTX 

type DBTX interface {
	Exec(context.Context, string, ...interface{}) (pgconn.CommandTag, error)
	Query(context.Context, string, ...interface{}) (pgx.Rows, error)
	QueryRow(context.Context, string, ...interface{}) pgx.Row
	CopyFrom(ctx context.Context, tableName pgx.Identifier, columnNames []string, rowSrc pgx.CopyFromSource) (int64, error)
}

type DelKHoleRangeParams 

type DelKHoleRangeParams struct {
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
	Start     int64  `json:"start"`
	Stop      int64  `json:"stop"`
}

type ExSymbol 

type ExSymbol struct {
	ID       int32  `json:"id"`
	Exchange string `json:"exchange"`
	ExgReal  string `json:"exg_real"`
	Market   string `json:"market"`
	Symbol   string `json:"symbol"`
	Combined bool   `json:"combined"`
	ListMs   int64  `json:"list_ms"`
	DelistMs int64  `json:"delist_ms"`
}

func GetAllExSymbols 

func GetAllExSymbols() []*ExSymbol

GetAllExSymbols Gets all the objects that have been loaded into the cache 获取已加载到缓存的所有标的

func GetExSymbol 

func GetExSymbol(exchange banexg.BanExchange, symbol string) (*ExSymbol, *errs.Error)

func GetExSymbolCur 

func GetExSymbolCur(symbol string) (*ExSymbol, *errs.Error)

func GetSymbolByID 

func GetSymbolByID(id int32) *ExSymbol

func ParseShort 

func ParseShort(exgName, short string) (*ExSymbol, *errs.Error)

func (*ExSymbol) GetValidStart 

func (s *ExSymbol) GetValidStart(startMS int64) int64

func (*ExSymbol) ToShort 

func (s *ExSymbol) ToShort() string

type FindKHolesArgs added in v0.1.12 

type FindKHolesArgs struct {
	Sid       int32
	TimeFrame string
	Start     int64
	Stop      int64
	Limit     int
	Offset    int
}

type GetKHolesParams 

type GetKHolesParams struct {
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
}

type InfoKline 

type InfoKline struct {
	*banexg.PairTFKline
	Adj      *AdjInfo
	IsWarmUp bool
}

type InsKline added in v0.1.5 

type InsKline struct {
	ID        int32  `json:"id"`
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
	StartMs   int64  `json:"start_ms"`
	StopMs    int64  `json:"stop_ms"`
}

type KHole 

type KHole struct {
	ID        int64  `json:"id"`
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
	Start     int64  `json:"start"`
	Stop      int64  `json:"stop"`
}

type KHoleExt 

type KHoleExt struct {
	*KHole
	TfMSecs int64
}

type KInfo 

type KInfo struct {
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
	Start     int64  `json:"start"`
	Stop      int64  `json:"stop"`
}

type KInfoExt 

type KInfoExt struct {
	KInfo
	TfMSecs int64
}

type KlineAgg 

type KlineAgg struct {
	TimeFrame string
	MSecs     int64
	Table     string
	AggFrom   string
	AggStart  string
	AggEnd    string
	AggEvery  string
	CpsBefore string
	Retention string
}

func GetKlineAggs 

func GetKlineAggs() []*KlineAgg

func NewKlineAgg 

func NewKlineAgg(TimeFrame, Table, AggFrom, AggStart, AggEnd, AggEvery, CpsBefore, Retention string) *KlineAgg

type KlineSid 

type KlineSid struct {
	banexg.Kline
	Sid int32
}

type KlineUn 

type KlineUn struct {
	Sid       int32   `json:"sid"`
	StartMs   int64   `json:"start_ms"`
	StopMs    int64   `json:"stop_ms"`
	Timeframe string  `json:"timeframe"`
	Open      float64 `json:"open"`
	High      float64 `json:"high"`
	Low       float64 `json:"low"`
	Close     float64 `json:"close"`
	Volume    float64 `json:"volume"`
	Info      float64 `json:"info"`
}

type PriceVol 

type PriceVol struct {
	Sid   int32
	Price float64
	Vol   float64
}

type Queries 

type Queries struct {
	// contains filtered or unexported fields
}

func Conn 

func Conn(ctx context.Context) (*Queries, *pgxpool.Conn, *errs.Error)

func New 

func New(db DBTX) *Queries

func (*Queries) AddAdjFactors 

func (q *Queries) AddAdjFactors(ctx context.Context, arg []AddAdjFactorsParams) (int64, error)

func (*Queries) AddCalendars 

func (q *Queries) AddCalendars(ctx context.Context, arg []AddCalendarsParams) (int64, error)

func (*Queries) AddInsJob added in v0.1.5 

func (q *Queries) AddInsJob(add AddInsKlineParams) (int32, *errs.Error)

func (*Queries) AddInsKline added in v0.1.5 

func (q *Queries) AddInsKline(ctx context.Context, arg AddInsKlineParams) (int32, error)

func (*Queries) AddKHoles 

func (q *Queries) AddKHoles(ctx context.Context, arg []AddKHolesParams) (int64, error)

func (*Queries) AddKInfo 

func (q *Queries) AddKInfo(ctx context.Context, arg AddKInfoParams) (*KInfo, error)

func (*Queries) AddSymbols 

func (q *Queries) AddSymbols(ctx context.Context, arg []AddSymbolsParams) (int64, error)

func (*Queries) CalcKLineRange 

func (q *Queries) CalcKLineRange(sid int32, timeFrame string, start, end int64) (int64, int64, *errs.Error)

CalcKLineRange Calculate the effective range of the specified period K-line within the specified range. 计算指定周期K线在指定范围内，有效区间。

func (*Queries) CalcKLineRanges 

func (q *Queries) CalcKLineRanges(timeFrame string, sids map[int32]bool) (map[int32][2]int64, *errs.Error)

func (*Queries) DelAdjFactors 

func (q *Queries) DelAdjFactors(ctx context.Context, sid int32) error

func (*Queries) DelFactors 

func (q *Queries) DelFactors(sid int32, startMS, endMS int64) *errs.Error

func (*Queries) DelInsKline added in v0.1.5 

func (q *Queries) DelInsKline(ctx context.Context, id int32) error

func (*Queries) DelKData added in v0.1.12 

func (q *Queries) DelKData(exs *ExSymbol, tfList []string, startMS, endMS int64) *errs.Error

func (*Queries) DelKHoleIDs 

func (q *Queries) DelKHoleIDs(ids ...int64) *errs.Error

func (*Queries) DelKHoleRange 

func (q *Queries) DelKHoleRange(ctx context.Context, arg DelKHoleRangeParams) error

func (*Queries) DelKHoles 

func (q *Queries) DelKHoles(sid int32, timeFrame string, startMS, endMS int64) *errs.Error

func (*Queries) DelKInfo 

func (q *Queries) DelKInfo(sid int32, timeFrame string) *errs.Error

func (*Queries) DelKLineUn 

func (q *Queries) DelKLineUn(sid int32, timeFrame string) *errs.Error

func (*Queries) DelKLines 

func (q *Queries) DelKLines(sid int32, timeFrame string, startMS, endMS int64) *errs.Error

func (*Queries) DownOHLCV2DB 

func (q *Queries) DownOHLCV2DB(exchange banexg.BanExchange, exs *ExSymbol, timeFrame string, startMS, endMS int64,
	pBar *utils.PrgBar) (int, *errs.Error)

DownOHLCV2DB Download K-line to database. This method should be called in a transaction, otherwise there will be errors in querying and updating related data. 下载K线到数据库，应在事务中调用此方法，否则查询更新相关数据会有错误

func (*Queries) Exec 

func (q *Queries) Exec(sql string, args ...interface{}) *errs.Error

func (*Queries) FindKHoles added in v0.1.12 

func (q *Queries) FindKHoles(args FindKHolesArgs) ([]*KHole, int64, *errs.Error)

func (*Queries) FindKInfos added in v0.1.12 

func (q *Queries) FindKInfos(ctx context.Context, sid int32) ([]*KInfo, error)

func (*Queries) FixKInfoZeros added in v0.1.12 

func (q *Queries) FixKInfoZeros() *errs.Error

FixKInfoZeros 修复kinfo表中start=0或stop=0的记录。通过查询实际K线数据范围来更新正确的start和stop值。

func (*Queries) GetAdjFactors 

func (q *Queries) GetAdjFactors(ctx context.Context, sid int32) ([]*AdjFactor, error)

func (*Queries) GetAdjOHLCV 

func (q *Queries) GetAdjOHLCV(adjs []*AdjInfo, timeFrame string, startMS, endMS int64, limit int, withUnFinish bool) ([]*banexg.Kline, *errs.Error)

GetAdjOHLCV Obtain K-line and weighted information (returns K-line that has not been weighted yet, needs to call ApplyAdj for weighted) 获取K线和复权信息（返回的是尚未复权的K线，需调用ApplyAdj复权）

func (*Queries) GetAdjs 

func (q *Queries) GetAdjs(sid int32) ([]*AdjInfo, *errs.Error)

func (*Queries) GetAllInsKlines added in v0.1.5 

func (q *Queries) GetAllInsKlines(ctx context.Context) ([]*InsKline, error)

func (*Queries) GetCalendars 

func (q *Queries) GetCalendars(name string, startMS, stopMS int64) ([][2]int64, *errs.Error)

func (*Queries) GetExSHoles 

func (q *Queries) GetExSHoles(exchange banexg.BanExchange, exs *ExSymbol, start, stop int64, full bool) ([][2]int64, *errs.Error)

GetExSHoles Retrieve all non trading time ranges for the specified Sid within a certain time period. For the 365 * 24 coin circle, it will not stop and return empty 获取指定Sid在某个时间段内，所有非交易时间范围。 对于币圈365*24不休，返回空

func (*Queries) GetInsKline added in v0.1.5 

func (q *Queries) GetInsKline(ctx context.Context, sid int32) (*InsKline, error)

func (*Queries) GetKHoles 

func (q *Queries) GetKHoles(ctx context.Context, arg GetKHolesParams) ([]*KHole, error)

func (*Queries) GetKlineRange 

func (q *Queries) GetKlineRange(sid int32, timeFrame string) (int64, int64)

func (*Queries) GetKlineRanges 

func (q *Queries) GetKlineRanges(sidList []int32, timeFrame string) map[int32][2]int64

func (*Queries) GetOHLCV 

func (q *Queries) GetOHLCV(exs *ExSymbol, timeFrame string, startMS, endMS int64, limit int, withUnFinish bool) ([]*AdjInfo, []*banexg.Kline, *errs.Error)

GetOHLCV Obtain the variety K-line, return the unweighted K-line and the weighting factor, and the caller can call ApplyAdj to re-weight 获取品种K线，返回未复权K线和复权因子，调用方可调用ApplyAdj进行复权

func (*Queries) InsertKLines 

func (q *Queries) InsertKLines(timeFrame string, sid int32, arr []*banexg.Kline) (int64, *errs.Error)

InsertKLines Only batch insert K-lines. To update associated information simultaneously, please use InsertKLinesAuto 只批量插入K线，如需同时更新关联信息，请使用InsertKLinesAuto

func (*Queries) InsertKLinesAuto 

func (q *Queries) InsertKLinesAuto(timeFrame string, sid int32, arr []*banexg.Kline, aggBig bool) (int64, *errs.Error)

InsertKLinesAuto Insert K-line into the database and call updateKRange to update associated information Before calling this method, it is necessary to determine whether it already exists in the database through GetKlineRange to avoid duplicate insertions 插入K线到数据库，同时调用UpdateKRange更新关联信息 调用此方法前必须通过GetKlineRange自行判断数据库中是否已存在，避免重复插入

func (*Queries) ListExchanges 

func (q *Queries) ListExchanges(ctx context.Context) ([]string, error)

func (*Queries) ListKHoles 

func (q *Queries) ListKHoles(ctx context.Context, dollar_1 []int32) ([]*KHole, error)

func (*Queries) ListKInfos 

func (q *Queries) ListKInfos(ctx context.Context) ([]*KInfo, error)

func (*Queries) ListSymbols 

func (q *Queries) ListSymbols(ctx context.Context, exchange string) ([]*ExSymbol, error)

func (*Queries) LoadExgSymbols 

func (q *Queries) LoadExgSymbols(exgName string) *errs.Error

func (*Queries) NewTx 

func (q *Queries) NewTx(ctx context.Context) (*Tx, *Queries, *errs.Error)

func (*Queries) PurgeKlineUn 

func (q *Queries) PurgeKlineUn() *errs.Error

func (*Queries) QueryOHLCV 

func (q *Queries) QueryOHLCV(sid int32, timeframe string, startMs, endMs int64, limit int, withUnFinish bool) ([]*banexg.Kline, *errs.Error)

func (*Queries) QueryOHLCVBatch 

func (q *Queries) QueryOHLCVBatch(sids []int32, timeframe string, startMs, endMs int64, limit int, handle func(int32, []*banexg.Kline)) *errs.Error

func (*Queries) SetCalendars 

func (q *Queries) SetCalendars(name string, items [][2]int64) *errs.Error

func (*Queries) SetKHole 

func (q *Queries) SetKHole(ctx context.Context, arg SetKHoleParams) error

func (*Queries) SetKInfo 

func (q *Queries) SetKInfo(ctx context.Context, arg SetKInfoParams) error

func (*Queries) SetListMS 

func (q *Queries) SetListMS(ctx context.Context, arg SetListMSParams) error

func (*Queries) UpdateKRange 

func (q *Queries) UpdateKRange(sid int32, timeFrame string, startMS, endMS int64, klines []*banexg.Kline, aggBig bool) *errs.Error

UpdateKRange 1. Update the effective range of the K-line 2. Search for holes and update Khole 3. Update continuous aggregation with larger cycles 1. 更新K线的有效区间 2. 搜索空洞，更新Khole 3. 更新更大周期的连续聚合

func (*Queries) UpdatePendingIns added in v0.1.5 

func (q *Queries) UpdatePendingIns() *errs.Error

UpdatePendingIns Update unfinished insertion tasks and call them when the robot starts, 更新未完成的插入任务，在机器人启动时调用，

func (*Queries) WithTx 

func (q *Queries) WithTx(tx pgx.Tx) *Queries

type SetKHoleParams 

type SetKHoleParams struct {
	ID    int64 `json:"id"`
	Start int64 `json:"start"`
	Stop  int64 `json:"stop"`
}

type SetKInfoParams 

type SetKInfoParams struct {
	Sid       int32  `json:"sid"`
	Timeframe string `json:"timeframe"`
	Start     int64  `json:"start"`
	Stop      int64  `json:"stop"`
}

type SetListMSParams 

type SetListMSParams struct {
	ID       int32 `json:"id"`
	ListMs   int64 `json:"list_ms"`
	DelistMs int64 `json:"delist_ms"`
}

type Tx 

type Tx struct {
	// contains filtered or unexported fields
}

func (*Tx) Close 

func (t *Tx) Close(ctx context.Context, commit bool) *errs.Error

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.