Documentation ¶
Overview ¶
Package db implements local price series database API.
Index ¶
- func Cleanup(ctx context.Context, dbPath, db string) error
- func DailyVolatility(rows []ResampledRow) (volatility float64, samples uint16)
- func PriceRowHeader() []string
- func ReadCSVTickers(r io.Reader, c *TickerRowConfig, tickers map[string]TickerRow) error
- func ResampledRowHeader() []string
- func TickerRowHeader() []string
- type Constraints
- func (c *Constraints) Category(cats ...string) *Constraints
- func (c *Constraints) CheckTicker(ticker string) bool
- func (c *Constraints) CheckTickerRow(r TickerRow) bool
- func (c *Constraints) Exchange(ex ...string) *Constraints
- func (c *Constraints) ExcludeTicker(tickers ...string) *Constraints
- func (c *Constraints) Industry(inds ...string) *Constraints
- func (c *Constraints) Name(names ...string) *Constraints
- func (c *Constraints) Sector(secs ...string) *Constraints
- func (c *Constraints) SetActive(active bool) *Constraints
- func (c *Constraints) Source(sources ...string) *Constraints
- func (c *Constraints) Ticker(tickers ...string) *Constraints
- type Date
- func DateInNY(now time.Time) Date
- func MaxDate(dates ...Date) Date
- func MinDate(dates ...Date) Date
- func NewDate(year uint16, month, day uint8) Date
- func NewDateFromString(s string) (Date, error)
- func NewDateFromTime(t time.Time) Date
- func NewDatetime(year uint16, month, day, hour, minute, second uint8, msec uint32) Date
- func (d Date) After(d2 Date) bool
- func (d Date) Before(d2 Date) bool
- func (d Date) Date() Date
- func (d Date) Day() uint8
- func (d Date) DaysInMonth() uint8
- func (d Date) Hour() uint8
- func (d Date) InRange(start, end Date) bool
- func (d *Date) InitMessage(js any) error
- func (d Date) IsLeapYear() bool
- func (d Date) IsZero() bool
- func (d Date) MarshalJSON() ([]byte, error)
- func (d Date) Millisecond() uint32
- func (d Date) Minute() uint8
- func (d Date) Monday() Date
- func (d Date) Month() uint8
- func (d Date) MonthStart() Date
- func (d Date) QuarterStart() Date
- func (d Date) Second() uint8
- func (d Date) String() string
- func (d Date) ToTime() time.Time
- func (d *Date) UnmarshalJSON(data []byte) error
- func (d Date) Year() uint16
- func (d Date) YearsTill(d2 Date) float64
- type Interval
- type IntradayRange
- type Metadata
- type PriceRow
- func (p PriceRow) Active() bool
- func (p PriceRow) CSV() []string
- func (p PriceRow) CloseUnadjusted() float32
- func (p PriceRow) HighFullyAdjusted() float32
- func (p PriceRow) HighSplitAdjusted() float32
- func (p PriceRow) LowFullyAdjusted() float32
- func (p PriceRow) LowSplitAdjusted() float32
- func (p PriceRow) OpenFullyAdjusted() float32
- func (p PriceRow) OpenSplitAdjusted() float32
- func (p *PriceRow) SetActive(active bool)
- type PriceRowConfig
- type Reader
- func (r *Reader) AllMonthlyRows() (map[string][]ResampledRow, error)
- func (r *Reader) AllTickerRows() (map[string]TickerRow, error)
- func (r *Reader) HasMetadata() bool
- func (r *Reader) HasMonthly() bool
- func (r *Reader) HasPrices(ticker string) bool
- func (r *Reader) HasTickers() bool
- func (r *Reader) InitMessage(js any) error
- func (r *Reader) Metadata() (Metadata, error)
- func (r *Reader) Monthly(ticker string, start, end Date) ([]ResampledRow, error)
- func (r *Reader) Prices(ticker string) ([]PriceRow, error)
- func (r *Reader) TickerRow(ticker string) (TickerRow, error)
- func (r *Reader) Tickers(ctx context.Context) ([]string, error)
- type ResampledRow
- type TickerRow
- type TickerRowConfig
- type TickerTableRow
- type Time
- type TimeOfDay
- func (t TimeOfDay) After(t2 TimeOfDay) bool
- func (t TimeOfDay) Before(t2 TimeOfDay) bool
- func (t TimeOfDay) Hour() uint8
- func (t *TimeOfDay) InitMessage(js any) error
- func (t TimeOfDay) IsZero() bool
- func (t TimeOfDay) Millisecond() uint32
- func (t TimeOfDay) Minute() uint8
- func (t TimeOfDay) Second() uint8
- func (t TimeOfDay) String() string
- type Writer
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Cleanup ¶ added in v0.1.9
Cleanup the DB: delete price files that do not have a corresponding ticker. This is useful, e.g. when a ticker gets renamed and its price series is downloaded under the new name, but the old series remains in the DB.
func DailyVolatility ¶ added in v0.0.8
func DailyVolatility(rows []ResampledRow) (volatility float64, samples uint16)
DailyVolatility computes the average daily absolute log-profit from the list of consecutive monthly bars.
func PriceRowHeader ¶ added in v0.0.10
func PriceRowHeader() []string
func ReadCSVTickers ¶ added in v0.1.9
ReadCSVTickers reads raw CSV and updates the tickers table compatible with DB Writer.
When config defines a header, CSV is assumed to be headless; otherwise the CSV file must have a header. In either case, the header must contain a Ticker column. Columns with an unrecognized header are ignored, missing columns are assumed to be of their default values, which are empty for all strings and true for Active.
func ResampledRowHeader ¶ added in v0.0.10
func ResampledRowHeader() []string
func TickerRowHeader ¶ added in v0.0.10
func TickerRowHeader() []string
TickerRowHeader for CSV table.
Types ¶
type Constraints ¶
type Constraints struct { Sources map[string]struct{} Tickers map[string]struct{} ExcludeTickers map[string]struct{} Exchanges map[string]struct{} Names map[string]struct{} Categories map[string]struct{} Sectors map[string]struct{} Industries map[string]struct{} Active *bool // optional constraint on whether ticker is active }
Constraints to filter the tickers and their time series. Zero value means no constraints.
func NewConstraints ¶
func NewConstraints() *Constraints
NewConstraints creates a new Constraints with no constraints.
func (*Constraints) Category ¶
func (c *Constraints) Category(cats ...string) *Constraints
Category adds categories to the constraints.
func (*Constraints) CheckTicker ¶
func (c *Constraints) CheckTicker(ticker string) bool
CheckTicker whether it satisfies the constraints.
func (*Constraints) CheckTickerRow ¶
func (c *Constraints) CheckTickerRow(r TickerRow) bool
CheckTickerRow whether it satisfies the constraints.
func (*Constraints) Exchange ¶
func (c *Constraints) Exchange(ex ...string) *Constraints
Exchange adds exchanges to the constraints.
func (*Constraints) ExcludeTicker ¶
func (c *Constraints) ExcludeTicker(tickers ...string) *Constraints
ExcludeTicker adds tickers to be ignored.
func (*Constraints) Industry ¶
func (c *Constraints) Industry(inds ...string) *Constraints
Industry adds industries to the constraints.
func (*Constraints) Name ¶
func (c *Constraints) Name(names ...string) *Constraints
Name adds a company name to the constraints.
func (*Constraints) Sector ¶
func (c *Constraints) Sector(secs ...string) *Constraints
Sector adds sectors to the constraints.
func (*Constraints) SetActive ¶ added in v0.4.0
func (c *Constraints) SetActive(active bool) *Constraints
SetActive bit whether the ticker must be listed or delisted.
func (*Constraints) Source ¶ added in v0.0.9
func (c *Constraints) Source(sources ...string) *Constraints
Source adds sources to the constraints.
func (*Constraints) Ticker ¶
func (c *Constraints) Ticker(tickers ...string) *Constraints
Ticker adds tickers to the constraints.
type Date ¶
Date records a calendar date and time as year, month, day and millisecond from the midnight. The struct is designed to fit into 8 bytes.
func NewDateFromString ¶
NewDateFromString creates a Date instance from a string representation.
func NewDateFromTime ¶
NewDateFromTime creates a Date instance from a time.Time value in UTC.
func NewDatetime ¶ added in v0.3.5
NewDatetime is a Date constructor with time of the day.
func (Date) DaysInMonth ¶ added in v0.0.8
DaysInMonth is the numder of days in the current month, which for February depends on the year.
func (Date) InRange ¶ added in v0.0.8
InRange checks if d is in the inclusive date range. Any of the bounds may be zero value, in which case it's ignored.
func (*Date) InitMessage ¶ added in v0.0.2
InitMessage implements message.Message.
func (Date) IsLeapYear ¶ added in v0.0.8
func (Date) MarshalJSON ¶
MarshalJSON implements json.Marshaler.
func (Date) Millisecond ¶ added in v0.3.5
func (Date) Monday ¶
Monday returns a new Date of the Monday midnight of the current date's week. Note: week is assumed to start on Sunday, so Monday(d=Sunday) returns the next day.
func (Date) MonthStart ¶
MonthStart returns the 1st of the month of the current date.
func (Date) QuarterStart ¶
QuarterStart returns the first day of the quarter of the current date.
func (*Date) UnmarshalJSON ¶
UnmarshalJSON implements json.Unmarshaler. NOTE: unlike other methods, this is a pointer method.
type Interval ¶ added in v0.0.8
type Interval struct { Min *float64 `json:"min"` Max *float64 `json:"max"` Start Date `json:"start"` End Date `json:"end"` }
Interval configures a set of constraints for a value over an optional time range. All interval ranges are inclusive, both value and time.
func (Interval) DateInRange ¶ added in v0.0.8
DateInRange checks if it belongs to the date range of the interval.
func (*Interval) InitMessage ¶ added in v0.0.8
func (Interval) ValueInRange ¶ added in v0.0.8
ValueInRange checks if it belongs to the interval.
type IntradayRange ¶ added in v0.4.0
IntradayRange configures inclusive time range within each trading day.
func (IntradayRange) InRange ¶ added in v0.4.0
func (i IntradayRange) InRange(t TimeOfDay) bool
func (*IntradayRange) InitMessage ¶ added in v0.4.0
func (i *IntradayRange) InitMessage(js any) error
type Metadata ¶
type Metadata struct { Start Date `json:"start"` // the earliest available price date End Date `json:"end"` // the latest available price date NumTickers int `json:"num_tickers"` NumPrices int `json:"num_prices"` // daily price samples NumMonthly int `json:"num_monthly"` // monthly price samples }
Metadata is the schema for the metadata.json file.
func (*Metadata) UpdateMonthly ¶ added in v0.1.9
func (m *Metadata) UpdateMonthly(monthly map[string][]ResampledRow)
func (*Metadata) UpdatePrices ¶ added in v0.1.9
func (*Metadata) UpdateTickers ¶ added in v0.1.9
type PriceRow ¶
type PriceRow struct { Date Date Open float32 // all other prices are unadjusted High float32 Low float32 Close float32 // unadjusted; negative means delisted CloseSplitAdjusted float32 // adjusted only for splits CloseFullyAdjusted float32 // adjusted for splits, dividends, spinoffs CashVolume float32 // shares volume * closing price }
PriceRow is a row in the prices table. It is intended for daily price points. Size: 32 bytes.
Note, that the sign of the (unadjusted) Close price indicates whether the ticker is listed at the close of the day (negate = delisted). Therefore, use CloseUnadjusted() and Active() methods to get the corresponding values.
Price and cash volume are assumed to be in the stock's native currency, e.g. dollar for the US stocks.
func ReadCSVPrices ¶ added in v0.1.9
func ReadCSVPrices(r io.Reader, c *PriceRowConfig) ([]PriceRow, error)
ReadCSVPrices reads raw CSV and creates a price series compatible with DB Writer.
When config defines a header, CSV is assumed to be headless; otherwise the CSV file must have a header. In either case, the header must contain at least one type of price. The remaining prices can be set by mapping them to the same column name. Columns with an unrecognized header are ignored, missing columns are set to 0.
func TestPrice ¶
TestPrice creates a PriceRow instance for use in tests. It uses the closing price to assign the other OHL prices.
func TestPriceRow ¶ added in v0.3.4
func TestPriceRow(date Date, open, high, low, close, closeSplitAdj, closeFullyAdj, dv float32, active bool) PriceRow
TestPriceRow is a complete version of TestPrice which allows to set all OHLC prices directly. The OHL prices are always fully adjusted.
func (PriceRow) CloseUnadjusted ¶
CloseUnadjusted price, separated from the activity status.
func (PriceRow) HighFullyAdjusted ¶ added in v0.3.3
func (PriceRow) HighSplitAdjusted ¶ added in v0.3.4
func (PriceRow) LowFullyAdjusted ¶ added in v0.3.3
func (PriceRow) LowSplitAdjusted ¶ added in v0.3.4
func (PriceRow) OpenFullyAdjusted ¶ added in v0.3.3
func (PriceRow) OpenSplitAdjusted ¶ added in v0.3.4
type PriceRowConfig ¶ added in v0.1.9
type PriceRowConfig struct { // Unadjusted Date string `json:"Date" default:"Date"` Open string `json:"Open" default:"Open"` High string `json:"High" default:"High"` Low string `json:"Low" default:"Low"` Close string `json:"Close" default:"Close"` Volume string `json:"Volume"` // Split adjusted OpenSplitAdjusted string `json:"Open split adj"` HighSplitAdjusted string `json:"High split adj"` LowSplitAdjusted string `json:"Low split adj"` CloseSplitAdjusted string `json:"Close split adj" default:"Close split adj"` VolumeSplitAdjusted string `json:"Volume split adj"` // Split & divided adjusted OpenFullyAdjusted string `json:"Open fully adj"` HighFullyAdjusted string `json:"High fully adj"` LowFullyAdjusted string `json:"Low fully adj"` CloseFullyAdjusted string `json:"Close fully adj" default:"Close fully adj"` VolumeFullyAdjusted string `json:"Volume fully adj"` // Adjustment independent values CashVolume string `json:"Cash Volume" default:"Cash Volume"` Active string `json:"Active" default:"Active"` Header []string `json:"header"` // for headless CSV }
PriceRowConfig sets the custom headers of input CSV file for price rows. The mapping is one-to-many; in particular, several price fields can map to the same CSV column.
Note, that the DB stores natively only unadjusted OHLC, split- and fully adjusted Close prices, and cash volume defined as close*<volume in shares>. When other fields are configured and imported, such as adjusted OHL and adjusted or unadjusted volume (in shares), the actual values stored in the DB are derived from the ratios of the corresponding Close prices.
func NewPriceRowConfig ¶ added in v0.1.9
func NewPriceRowConfig() *PriceRowConfig
func (*PriceRowConfig) HasPrice ¶ added in v0.1.9
func (c *PriceRowConfig) HasPrice(header []string) bool
HasPrice checks that the header contains at least one closing price column.
func (*PriceRowConfig) InitMessage ¶ added in v0.1.9
func (c *PriceRowConfig) InitMessage(js any) error
func (*PriceRowConfig) MapColumns ¶ added in v0.1.9
func (c *PriceRowConfig) MapColumns(header []string) [][]priceField
MapColumns maps the i'th header column to the list of PriceRow fields. In particular, a single price column can map to several price fields in PriceRow.
type Reader ¶ added in v0.0.3
type Reader struct { DBPath string `json:"DB path"` // default: ~/.stockparfait DB string `json:"DB" required:"true"` // specific DB in path UseTickers []string `json:"tickers"` ExcludeTickers []string `json:"exclude tickers"` Sources []string `json:"sources"` Exchanges []string `json:"exchanges"` Names []string `json:"names"` Categories []string `json:"categories"` Sectors []string `json:"sectors"` Industries []string `json:"industries"` Start Date `json:"start"` End Date `json:"end"` Active *bool `json:"active"` YearlyGrowth *Interval `json:"yearly growth"` CashVolume *Interval `json:"cash volume"` Volatility *Interval `json:"volatility"` Intraday *IntradayRange `json:"intraday"` // contains filtered or unexported fields }
Reader of the database, which also implements message.Message and can be used directly in a config.
func (*Reader) AllMonthlyRows ¶ added in v0.1.9
func (r *Reader) AllMonthlyRows() (map[string][]ResampledRow, error)
AllMonthlyRows returns all the monthly resampled rows from the DB as a {ticker -> row} map, compatible with Writer.WriteMonthly() method. Note: modifying the map will modify the Reader's cached copy.
func (*Reader) AllTickerRows ¶ added in v0.1.9
AllTickerRows returns all the ticker rows from the DB as a {ticker -> row} map, compatible with Writer.WriteTickers() method. Note: modifying the map will modify the Reader's cached copy.
func (*Reader) HasMetadata ¶ added in v0.1.9
HasMetadata checks if the DB exists and has the metadata.
func (*Reader) HasMonthly ¶ added in v0.1.9
HasMonthly checks if the DB exists and has the monthly table.
func (*Reader) HasPrices ¶ added in v0.1.9
HasPrices checks if the DB exists and has the prices for the ticker.
func (*Reader) HasTickers ¶ added in v0.1.9
HasTickers checks if the DB exists and has the tickers table.
func (*Reader) InitMessage ¶ added in v0.0.6
InitMessage implements message.Message.
func (*Reader) Metadata ¶ added in v0.0.3
Metadata for the database. It is cached in memory upon the first call.
func (*Reader) Monthly ¶ added in v0.0.3
func (r *Reader) Monthly(ticker string, start, end Date) ([]ResampledRow, error)
Monthly price data for ticker within the inclusive date range, sorted by date. If any of start or end is zero value, the corresponding Reader constraint is used. Data for all tickers are cached in memory upon the first call. Go routine safe assuming constraints are not modified.
func (*Reader) Prices ¶ added in v0.0.3
Prices for ticker satilfying Reader's constraints, sorted by date. Prices are NOT cached in memory, and are read from disk every time. It is probably go routine safe, if the underlying OS allows to open and read the same file multiple times from the same process. Reading different tickers is definitely safe in parallel, assuming consraints are not modified.
func (*Reader) TickerRow ¶ added in v0.0.3
TickerRow for the given ticker. It's an error if a ticker is not in R. Tickers are cached in memory upon the first call. Go routine safe.
func (*Reader) Tickers ¶ added in v0.0.3
Tickers returns the list of tickers satisfying current Reader's constraints. All tickers are cached in memory, and tickers are filtered for each call. Therefore, modifying Reader's constraints takes effect at the next call without re-reading the tickers. Go-routine safe assuming constraints are not modified.
type ResampledRow ¶
type ResampledRow struct { Open float32 OpenSplitAdjusted float32 OpenFullyAdjusted float32 Close float32 // unadjusted CloseSplitAdjusted float32 // adjusted only for splits CloseFullyAdjusted float32 // adjusted for splits, dividends, spinoffs CashVolume float32 DateOpen Date DateClose Date // Sum of absolute values of daily log-profits within the bar: // sum(abs(log(p(t+1))-log(p(t)))). Note, that it always has NumSamples-1 // samples. When summing over multiple resampled bars, recover the missing // sample as abs(log(open(t))-log(close(t-1))). SumAbsLogProfits float32 NumSamples uint16 Active bool // if ticker is active at bar's close }
ResampledRow is a multi-day bar with some additional daily statistics. Size: 44 bytes.
func ComputeMonthly ¶
func ComputeMonthly(prices []PriceRow) []ResampledRow
ComputeMonthly converts daily price series into resampled monthly price series.
func TestResampled ¶
func TestResampled(dateOpen, dateClose Date, open, close, adj, dv float32, active bool) ResampledRow
TestResampled creates a new ResampledRow for use in tests.
func (ResampledRow) CSV ¶ added in v0.0.10
func (r ResampledRow) CSV() []string
type TickerRow ¶
type TickerRow struct { Source string // where the ticker was downloaded, e.g. NDL table name Exchange string // the primary exchange trading this ticker Name string // the company name Category string Sector string Industry string Location string SECFilings string // URL CompanySite string // URL Active bool // ticker is listed at the last price date }
TickerRow is a row in the tickers table.
type TickerRowConfig ¶ added in v0.1.9
type TickerRowConfig struct { Ticker string `json:"Ticker" default:"Ticker"` Source string `json:"Source" default:"Source"` Exchange string `json:"Exchange" default:"Exchange"` Name string `json:"Name" default:"Name"` Category string `json:"Category" default:"Category"` Sector string `json:"Sector" default:"Sector"` Industry string `json:"Industry" default:"Industry"` Location string `json:"Location" default:"Location"` SECFilings string `json:"SEC Filings" default:"SEC Filings"` CompanySite string `json:"Company Site" default:"Company Site"` Active string `json:"Active" default:"Active"` Header []string `json:"header"` // for headless CSV }
TickerRowConfig sets the custom headers of input CSV file for ticker rows.
func NewTickerRowConfig ¶ added in v0.1.9
func NewTickerRowConfig() *TickerRowConfig
func (*TickerRowConfig) HasTicker ¶ added in v0.1.9
func (c *TickerRowConfig) HasTicker(header []string) bool
HasTicker checks the header for the column corresponding to the ticker name.
func (*TickerRowConfig) InitMessage ¶ added in v0.1.9
func (c *TickerRowConfig) InitMessage(js any) error
InitMessage implements message.Message.
func (*TickerRowConfig) MapColumns ¶ added in v0.1.9
func (c *TickerRowConfig) MapColumns(header []string) []tickerField
MapColumns maps the i'th header column to the j'th TickerTableRow field. Headers that don't match any configured column are mapped to -1.
type TickerTableRow ¶ added in v0.1.9
TickerTableRow implements table.Row by adding ticker name to TickerRow.
func (TickerTableRow) CSV ¶ added in v0.1.9
func (r TickerTableRow) CSV() []string
type Time ¶
Time is a wrapper around time.Time with JSON methods.
func (*Time) MarshalJSON ¶
MarshalJSON implements json.Marshaler.
func (*Time) UnmarshalJSON ¶
UnmarshalJSON implements json.Unmarshaler.
type TimeOfDay ¶ added in v0.4.0
type TimeOfDay uint32
TimeOfDay as milliseconds from midnight.
func NewTimeOfDay ¶ added in v0.4.0
NewTimeOfDay creates TimeOfDay out of components. It panics if any of the arguments is out of range: hour>23, minute, second>59 or msec>999.
func NewTimeOfDayFromString ¶ added in v0.4.0
NewTimeOfDayFromString parses a string "hh:mm:ss.xxx" with optional seconds and milliseconds.
func (*TimeOfDay) InitMessage ¶ added in v0.4.0
func (TimeOfDay) Millisecond ¶ added in v0.4.0
type Writer ¶ added in v0.0.3
type Writer struct { Metadata Metadata // contains filtered or unexported fields }
func (*Writer) WriteMetadata ¶ added in v0.0.3
WriteMetadata saves the metadata accumulated by the Write* methods. It is stored in JSON format to be human-readable.
func (*Writer) WriteMonthly ¶ added in v0.0.3
func (w *Writer) WriteMonthly(monthly map[string][]ResampledRow) error
WriteMonthly saves the monthly resampled table to the DB file and sets the number of samples in the metadata. ResampledRow's are indexed by ticker, and for each ticker are assumed to be sorted by the closing date.
func (*Writer) WritePrices ¶ added in v0.0.3
WritePrices saves the ticker prices to the DB file and incrementally updates the metadata. Prices are assumed to be sorted by date.