README
¶
MarketStore frontend API
Transport Methods
MarketStore communicates with its clients through standard HTTP in Messagepack RPC (Messagepack version of JSON-RPC 2.0).
DataService.ListSymbols()
Input
no parameters
Output
list of string for each unique symbol stored in the server.
DataService.Query()
Input
Query() interface accepts a list of "requests", each of which is a map with the following fields.
-
destination (
string
)A string path of the query target. A TimeBucketKey contains a Symbol, Timeframe, and an AttributeGroup. For example, "TSLA/1Min/OHLCV" is an example TimeBucketKey. In this example, TSLA is the Symbol, 1Min is the TimeFrame, and OHLCV is the AttributeGroup. Moreover, a single destination can include multiple symbols split by commas for a multi-symbol query. For example, "TSLA,F,NVDA/1Min/OHLCV" will query data for Symbols TSLA, F, and NVDA all across the same TimeFrame, AttributeGroup.
-
epoch_start (
int64
)An integer epoch seconds from Unix epoch time. Rows timestamped equal to or after this time will be returned.
-
epoch_end (
int64
)An integer epoch seconds from Unix epoch time. Rows timestamped equal to or before this time will be returned.
-
limit_record_count (
int
)An integer to limit the number of rows to be returned from the query.
-
limit_from_start (
bool
)A boolean value to indicate if limit_recourd_count should be counted from the lower side of result set or upper. Default to false, meaning from the upper.
Note: It is also possible to query multiple TimeBucketKeys at once. The requests parameter is passed a list of query structures (See examples).
Output
The output returns the same number of "responses" as the requests, each of which has the following fields.
-
result
A MultiDataset type. See below for this type.
DataService.Write()
Input
-
dataset
A MultiDataset type. See below for this type.
-
is_variable_length (
bool
)A boolean value for telling MarketStore if the write procedure will be dynamic in length.
Output
The API will return an empty response on success. Should the write call fail, the response will include the original input as well as an error returned by the server.
MultiDataset type
This is the common wire format to represent a series of columns containing multiple slices (horizontal partitions). It is a map with the following fields that represents set of column-oriented data
-
types (
[]string
)a list of strings for the column types compatible with numpy dtypes (e.g., 'i4', 'f8')
-
names (
[]string
)a list of strings for the column names
-
data (
[][]byte
)a list of byte arrays with each being the binary column data
-
startindex (
[]int
)a list of integer to indicate which element each slice starts at
-
lengths (
[]int
)a list of integer to indicate how many elements each slice has
Documentation
¶
Index ¶
- Variables
- func Heartbeat(address string)
- func NewServer() (*RpcServer, *DataService)
- type DataService
- func (s *DataService) Init()
- func (s *DataService) ListSymbols(r *http.Request, args *ListSymbolsArgs, response *ListSymbolsReply) (err error)
- func (s *DataService) Query(r *http.Request, reqs *MultiQueryRequest, response *MultiQueryResponse) (err error)
- func (s *DataService) Write(r *http.Request, reqs *MultiWriteRequest, response *MultiWriteResponse) (err error)
- type HeartbeatMessage
- type ListSymbolsArgs
- type ListSymbolsReply
- type MultiQueryRequest
- type MultiQueryResponse
- type MultiWriteRequest
- type MultiWriteResponse
- type QueryRequest
- type QueryRequestBuilder
- func (b *QueryRequestBuilder) End() QueryRequest
- func (b *QueryRequestBuilder) EpochEnd(value int64) *QueryRequestBuilder
- func (b *QueryRequestBuilder) EpochStart(value int64) *QueryRequestBuilder
- func (b *QueryRequestBuilder) Functions(value []string) *QueryRequestBuilder
- func (b *QueryRequestBuilder) LimitFromStart(value bool) *QueryRequestBuilder
- func (b *QueryRequestBuilder) LimitRecordCount(value int) *QueryRequestBuilder
- type QueryResponse
- type RpcServer
- type WriteRequest
- type WriteResponse
Constants ¶
This section is empty.
Variables ¶
var Queryable uint32 // treated as bool
Functions ¶
func NewServer ¶
func NewServer() (*RpcServer, *DataService)
Types ¶
type DataService ¶
type DataService struct{}
func (*DataService) Init ¶
func (s *DataService) Init()
func (*DataService) ListSymbols ¶
func (s *DataService) ListSymbols(r *http.Request, args *ListSymbolsArgs, response *ListSymbolsReply) (err error)
func (*DataService) Query ¶
func (s *DataService) Query(r *http.Request, reqs *MultiQueryRequest, response *MultiQueryResponse) (err error)
func (*DataService) Write ¶
func (s *DataService) Write(r *http.Request, reqs *MultiWriteRequest, response *MultiWriteResponse) (err error)
type HeartbeatMessage ¶
type ListSymbolsArgs ¶
type ListSymbolsArgs struct{}
type ListSymbolsReply ¶
type ListSymbolsReply struct {
Results []string
}
type MultiQueryRequest ¶
type MultiQueryRequest struct { /* A multi-request allows for different Timeframes and record formats for each request */ Requests []QueryRequest `msgpack:"requests"` }
type MultiQueryResponse ¶
type MultiQueryResponse struct { Responses []QueryResponse `msgpack:"responses"` Version string `msgpack:"version"` // Server Version Timezone string `msgpack:"timezone"` // Server Timezone }
type MultiWriteRequest ¶
type MultiWriteRequest struct { /* A multi-request allows for different Timeframes and record formats for each request */ Requests []WriteRequest `msgpack:"requests"` }
type MultiWriteResponse ¶
type MultiWriteResponse struct {
Responses []WriteResponse `msgpack:"responses"`
}
type QueryRequest ¶
type QueryRequest struct { // Note: SQL is not fully supported IsSQLStatement bool `msgpack:"is_sqlstatement"` // If this is a SQL request, Only SQLStatement is relevant SQLStatement string `msgpack:"sql_statement"` // Destination is <symbol>/<timeframe>/<attributegroup> Destination string `msgpack:"destination"` // This is not usually set, defaults to Symbol/Timeframe/AttributeGroup KeyCategory string `msgpack:"key_category,omitempty"` // Lower time predicate (i.e. index >= start) in unix epoch second EpochStart *int64 `msgpack:"epoch_start,omitempty"` // Upper time predicate (i.e. index <= end) in unix epoch second EpochEnd *int64 `msgpack:"epoch_end,omitempty"` // Number of max returned rows from lower/upper bound LimitRecordCount *int `msgpack:"limit_record_count,omitempty"` // Set to true if LimitRecordCount should be from the lower LimitFromStart *bool `msgpack:"limit_from_start,omitempty"` // Support for functions is experimental and subject to change Functions []string `msgpack:"functions,omitempty"` }
This is the parameter interface for DataService.Query method.
type QueryRequestBuilder ¶
type QueryRequestBuilder struct {
// contains filtered or unexported fields
}
QueryRequestBuilder is a builder for QueryRequest to set various parameters flexibly.
qr := NewQueryRequestBuilder("TSLA/1D/OHLCV").LimitRecourdCount(100).End()
func NewQueryRequestBuilder ¶
func NewQueryRequestBuilder(destination string) *QueryRequestBuilder
func (*QueryRequestBuilder) End ¶
func (b *QueryRequestBuilder) End() QueryRequest
func (*QueryRequestBuilder) EpochEnd ¶
func (b *QueryRequestBuilder) EpochEnd(value int64) *QueryRequestBuilder
func (*QueryRequestBuilder) EpochStart ¶
func (b *QueryRequestBuilder) EpochStart(value int64) *QueryRequestBuilder
func (*QueryRequestBuilder) Functions ¶
func (b *QueryRequestBuilder) Functions(value []string) *QueryRequestBuilder
func (*QueryRequestBuilder) LimitFromStart ¶
func (b *QueryRequestBuilder) LimitFromStart(value bool) *QueryRequestBuilder
func (*QueryRequestBuilder) LimitRecordCount ¶
func (b *QueryRequestBuilder) LimitRecordCount(value int) *QueryRequestBuilder
type QueryResponse ¶
type QueryResponse struct {
Result *io.NumpyMultiDataset `msgpack:"result"`
}
type WriteRequest ¶
type WriteRequest struct { Data *io.NumpyMultiDataset `msgpack:"dataset"` IsVariableLength bool `msgpack:"is_variable_length"` }