Documentation ¶
Overview ¶
Package etable provides the etable.Table structure which provides a DataTable or DataFrame data representation, which is a collection of columnar data all having the same number of rows.
Each column is an etensor.Tensor, so it can represent scalar or higher dimensional data per each cell (row x column location) in the Table. Thus, scalar data is represented using a 1D Tensor where the 1 dimension is the rows of the table, and likewise higher dimensional data always has the outer-most dimension as the row.
All tensors MUST have RowMajor stride layout for consistency, with the outer-most dimension as the row dimension, which is enforced to be the same across all columns.
The tensor columns can be individually converted to / from arrow.Tensors and conversion between arrow.Table is planned, along with inter-conversion with relevant gonum structures including the planned dframe.Frame.
Native support is provided for basic CSV, TSV I/O, including the C++ emergent standard TSV format with full type information in the first row column headers.
The etable.IndexView is an indexed view into a Table, which is used for all data-processing operations such as Sort, Filter, Split (groupby), and for aggregating data as in a pivot-table.
See etable/agg package for aggregation functions that operate on the IndexView to perform standard aggregation operations such as Sum, Mean, etc.
Other relevant examples of DataTable-like structures: * https://github.com/apache/arrow/tree/master/go/arrow Table * http://xarray.pydata.org/en/stable/index.html * https://pandas.pydata.org/pandas-docs/stable/reference/frame.html * https://www.rdocumentation.org/packages/base/versions/3.4.3/topics/data.frame * https://github.com/tobgu/qframe * https://github.com/kniren/gota
Index ¶
- Constants
- Variables
- func DetectEmerHeaders(hdrs []string) bool
- func EmerColType(nm string) (etensor.Type, string)
- func FilterNull(et *Table, row int) bool
- func InferDataType(str string) etensor.Type
- func ShapeFromString(dims string) []int
- type Column
- type Delims
- func (i Delims) Desc() string
- func (i Delims) Int64() int64
- func (i Delims) MarshalText() ([]byte, error)
- func (dl Delims) Rune() rune
- func (i *Delims) SetInt64(in int64)
- func (i *Delims) SetString(s string) error
- func (i Delims) String() string
- func (i *Delims) UnmarshalText(text []byte) error
- func (i Delims) Values() []enums.Enum
- type FilterFunc
- type IndexView
- func (ix *IndexView) AddIndex(idx int)
- func (ix *IndexView) AddRows(n int)
- func (ix *IndexView) AggCol(colIndex int, ini float64, fun etensor.AggFunc) []float64
- func (ix *IndexView) Clone() *IndexView
- func (ix *IndexView) CopyFrom(oix *IndexView)
- func (ix *IndexView) DeleteInvalid()
- func (ix *IndexView) DeleteRows(at, n int)
- func (ix *IndexView) Filter(filterFunc func(et *Table, row int) bool)
- func (ix *IndexView) FilterCol(colIndex int, str string, exclude, contains, ignoreCase bool)
- func (ix *IndexView) FilterColName(colNm string, str string, exclude, contains, ignoreCase bool) error
- func (ix *IndexView) InsertRows(at, n int)
- func (ix *IndexView) Len() int
- func (ix *IndexView) Less(i, j int) bool
- func (ix *IndexView) NewTable() *Table
- func (ix *IndexView) OpenCSV(filename core.Filename, delim Delims) error
- func (ix *IndexView) OpenFS(fsys fs.FS, filename string, delim Delims) error
- func (ix *IndexView) Permuted()
- func (ix *IndexView) RowsByString(colNm string, str string, contains, ignoreCase bool) []int
- func (ix *IndexView) RowsByStringIndex(colIndex int, str string, contains, ignoreCase bool) []int
- func (ix *IndexView) RowsByStringTry(colNm string, str string, contains, ignoreCase bool) ([]int, error)
- func (ix *IndexView) SaveCSV(filename core.Filename, delim Delims, headers bool) error
- func (ix *IndexView) Sequential()
- func (ix *IndexView) SetTable(et *Table)
- func (ix *IndexView) Sort(lessFunc func(et *Table, i, j int) bool)
- func (ix *IndexView) SortCol(colIndex int, ascending bool)
- func (ix *IndexView) SortColName(colNm string, ascending bool) error
- func (ix *IndexView) SortColNames(colNms []string, ascending bool) error
- func (ix *IndexView) SortCols(colIndexes []int, ascending bool)
- func (ix *IndexView) SortIndexes()
- func (ix *IndexView) SortStable(lessFunc func(et *Table, i, j int) bool)
- func (ix *IndexView) SortStableCol(colIndex int, ascending bool)
- func (ix *IndexView) SortStableColName(colNm string, ascending bool) error
- func (ix *IndexView) SortStableColNames(colNms []string, ascending bool) error
- func (ix *IndexView) SortStableCols(colIndexes []int, ascending bool)
- func (ix *IndexView) Swap(i, j int)
- func (ix *IndexView) WriteCSV(w io.Writer, delim Delims, headers bool) error
- type LessFunc
- type Schema
- type SplitAgg
- type Splits
- func (spl *Splits) AddAgg(name string, colIndex int) *SplitAgg
- func (spl *Splits) AggByColName(name string) *SplitAgg
- func (spl *Splits) AggByColNameTry(name string) (*SplitAgg, error)
- func (spl *Splits) AggByName(name string) *SplitAgg
- func (spl *Splits) AggByNameTry(name string) (*SplitAgg, error)
- func (spl *Splits) AggsToTable(colName bool) *Table
- func (spl *Splits) AggsToTableCopy(colName bool) *Table
- func (spl *Splits) ByValue(values []string) []int
- func (spl *Splits) Clone() *Splits
- func (spl *Splits) CopyFrom(osp *Splits)
- func (spl *Splits) Delete(idx int)
- func (spl *Splits) DeleteAggs()
- func (spl *Splits) ExtractLevels(levels []int) (*Splits, error)
- func (spl *Splits) Filter(fun func(idx int) bool)
- func (spl *Splits) Len() int
- func (spl *Splits) Less(i, j int) bool
- func (spl *Splits) New(dt *Table, values []string, rows ...int) *IndexView
- func (spl *Splits) ReorderLevels(order []int) error
- func (spl *Splits) SetLevels(levels ...string)
- func (spl *Splits) Sort(lessFunc func(spl *Splits, i, j int) bool)
- func (spl *Splits) SortLevels()
- func (spl *Splits) SortOrder(order []int) error
- func (spl *Splits) Swap(i, j int)
- func (spl *Splits) Table() *Table
- type SplitsLessFunc
- type Table
- func (dt *Table) AddCol(tsr etensor.Tensor, name string) error
- func (dt *Table) AddRows(n int)
- func (dt *Table) AppendRows(dt2 *Table)
- func (dt *Table) CellFloat(colNm string, row int) float64
- func (dt *Table) CellFloatIndex(col, row int) float64
- func (dt *Table) CellFloatTry(colNm string, row int) (float64, error)
- func (dt *Table) CellString(colNm string, row int) string
- func (dt *Table) CellStringIndex(col, row int) string
- func (dt *Table) CellStringTry(colNm string, row int) (string, error)
- func (dt *Table) CellTensor(colNm string, row int) etensor.Tensor
- func (dt *Table) CellTensorFloat1D(colNm string, row int, idx int) float64
- func (dt *Table) CellTensorFloat1DTry(colNm string, row int, idx int) (float64, error)
- func (dt *Table) CellTensorIndex(col, row int) etensor.Tensor
- func (dt *Table) CellTensorTry(colNm string, row int) (etensor.Tensor, error)
- func (dt *Table) Clone() *Table
- func (dt *Table) Col(i int) etensor.Tensor
- func (dt *Table) ColByName(name string) etensor.Tensor
- func (dt *Table) ColByNameTry(name string) (etensor.Tensor, error)
- func (dt *Table) ColIndex(name string) int
- func (dt *Table) ColIndexTry(name string) (int, error)
- func (dt *Table) ColIndexesByNames(names []string) []int
- func (dt *Table) ColIndexesByNamesTry(names []string) ([]int, error)
- func (dt *Table) ColName(i int) string
- func (dt *Table) CopyCell(colNm string, row int, cpt *Table, cpColNm string, cpRow int) error
- func (dt *Table) CopyMetaDataFrom(cp *Table)
- func (dt *Table) DeleteAll()
- func (dt *Table) DeleteColIndex(idx int)
- func (dt *Table) DeleteColName(name string) error
- func (dt *Table) EmerHeaders() []string
- func (dt *Table) IsValidRow(row int) bool
- func (dt *Table) IsValidRowTry(row int) error
- func (dt *Table) NumCols() int
- func (dt *Table) NumRows() int
- func (dt *Table) OpenCSV(filename core.Filename, delim Delims) error
- func (dt *Table) OpenFS(fsys fs.FS, filename string, delim Delims) error
- func (dt *Table) ReadCSV(r io.Reader, delim Delims) error
- func (dt *Table) ReadCSVRow(rec []string, row int)
- func (dt *Table) RowsByString(colNm string, str string, contains, ignoreCase bool) []int
- func (dt *Table) RowsByStringIndex(colIndex int, str string, contains, ignoreCase bool) []int
- func (dt *Table) RowsByStringTry(colNm string, str string, contains, ignoreCase bool) ([]int, error)
- func (dt *Table) SaveCSV(filename core.Filename, delim Delims, headers bool) error
- func (dt *Table) Schema() Schema
- func (dt *Table) SetCellFloat(colNm string, row int, val float64) bool
- func (dt *Table) SetCellFloatIndex(col, row int, val float64) bool
- func (dt *Table) SetCellFloatTry(colNm string, row int, val float64) error
- func (dt *Table) SetCellString(colNm string, row int, val string) bool
- func (dt *Table) SetCellStringIndex(col, row int, val string) bool
- func (dt *Table) SetCellStringTry(colNm string, row int, val string) error
- func (dt *Table) SetCellTensor(colNm string, row int, val etensor.Tensor) bool
- func (dt *Table) SetCellTensorFloat1D(colNm string, row int, idx int, val float64) bool
- func (dt *Table) SetCellTensorFloat1DTry(colNm string, row int, idx int, val float64) error
- func (dt *Table) SetCellTensorIndex(col, row int, val etensor.Tensor) bool
- func (dt *Table) SetCellTensorTry(colNm string, row int, val etensor.Tensor) error
- func (dt *Table) SetFromSchema(sc Schema, rows int)
- func (dt *Table) SetMetaData(key, val string)
- func (dt *Table) SetNumRows(rows int)
- func (dt *Table) UpdateColNameMap()
- func (dt *Table) WriteCSV(w io.Writer, delim Delims, headers bool) error
- func (dt *Table) WriteCSVHeaders(w io.Writer, delim Delims) (int, error)
- func (dt *Table) WriteCSVRow(w io.Writer, row int, delim Delims) error
- func (dt *Table) WriteCSVRowWriter(cw *csv.Writer, row int, ncol int) error
Constants ¶
const ( // Contains means the string only needs to contain the target string (see Equals) Contains bool = true // Equals means the string must equal the target string (see Contains) Equals = false // IgnoreCase means that differences in case are ignored in comparing strings IgnoreCase = true // UseCase means that case matters when comparing strings UseCase = false )
Named arg values for Contains, IgnoreCase
const ( // Ascending specifies an ascending sort direction for etable Sort routines Ascending = true // Descending specifies a descending sort direction for etable Sort routines Descending = false )
const ( // Headers is passed to CSV methods for the headers arg, to use headers Headers = true // NoHeaders is passed to CSV methods for the headers arg, to not use headers NoHeaders = false )
const ( // ColNameOnly means resulting agg table just has the original column name, no aggregation name ColNameOnly bool = true // AddAggName means resulting agg table columns have aggregation name appended AddAggName = false )
use these for arg to ArgsToTable*
Variables ¶
var EmerHdrCharToType = map[byte]etensor.Type{ '$': etensor.STRING, '%': etensor.FLOAT32, '#': etensor.FLOAT64, '|': etensor.INT64, '@': etensor.UINT8, '&': etensor.STRING, '^': etensor.BOOL, }
var EmerHdrTypeToChar map[etensor.Type]byte
Functions ¶
func DetectEmerHeaders ¶
DetectEmerHeaders looks for emergent header special characters -- returns true if found
func EmerColType ¶
EmerColType parses the column header for type information using the emergent naming convention
func FilterNull ¶
FilterNull is a FilterFunc that filters out all rows that have a Null value in a 1D (scalar) column, according to the IsNull flag
func InferDataType ¶
InferDataType returns the inferred data type for the given string only deals with float64, int, and string types
func ShapeFromString ¶
ShapeFromString parses string representation of shape as N:d,d,..
Types ¶
type Column ¶
type Column struct { // name of column -- must be unique for a table Name string // data type, using etensor types which are isomorphic with arrow.Type Type etensor.Type // shape of a single cell in the column (i.e., without the row dimension) -- for scalars this is nil -- tensor column will add the outer row dimension to this shape CellShape []int // names of the dimensions within the CellShape -- 'Row' will be added to outer dimension DimNames []string }
Column specifies everything about a column -- can be used for constructing tables
type Delims ¶
type Delims int32 //enums:enum
Delim are standard CSV delimiter options (Tab, Comma, Space)
const ( // Tab is the tab rune delimiter, for TSV tab separated values Tab Delims = iota // Comma is the comma rune delimiter, for CSV comma separated values Comma // Space is the space rune delimiter, for SSV space separated value Space // Detect is used during reading a file -- reads the first line and detects tabs or commas Detect )
const DelimsN Delims = 4
DelimsN is the highest valid value for type Delims, plus one.
func DelimsValues ¶
func DelimsValues() []Delims
DelimsValues returns all possible values for the type Delims.
func (Delims) MarshalText ¶
MarshalText implements the encoding.TextMarshaler interface.
func (*Delims) SetString ¶
SetString sets the Delims value from its string representation, and returns an error if the string is invalid.
func (*Delims) UnmarshalText ¶
UnmarshalText implements the encoding.TextUnmarshaler interface.
type FilterFunc ¶
FilterFunc is a function used for filtering that returns true if Table row should be included in the current filtered view of the table, and false if it should be removed.
type IndexView ¶
type IndexView struct { // Table that we are an indexed view onto Table *Table // current indexes into Table Indexes []int // contains filtered or unexported fields }
IndexView is an indexed wrapper around an etable.Table that provides a specific view onto the Table defined by the set of indexes. This provides an efficient way of sorting and filtering a table by only updating the indexes while doing nothing to the Table itself. To produce a table that has data actually organized according to the indexed order, call the NewTable method. IndexView views on a table can also be organized together as Splits of the table rows, e.g., by grouping values along a given column.
func NewIndexView ¶
NewIndexView returns a new IndexView based on given table, initialized with sequential idxes
func (*IndexView) AddRows ¶
AddRows adds n rows to end of underlying Table, and to the indexes in this view
func (*IndexView) AggCol ¶
AggCol applies given aggregation function to each element in the given column, using float64 conversions of the values. init is the initial value for the agg variable. Operates independently over each cell on n-dimensional columns and returns the result as a slice of values per cell.
func (*IndexView) CopyFrom ¶
CopyFrom copies from given other IndexView (we have our own unique copy of indexes)
func (*IndexView) DeleteInvalid ¶
func (ix *IndexView) DeleteInvalid()
DeleteInvalid deletes all invalid indexes from the list. Call this if rows (could) have been deleted from table.
func (*IndexView) DeleteRows ¶
DeleteRows deletes n rows of indexes starting at given index in the list of indexes
func (*IndexView) Filter ¶
Filter filters the indexes into our Table using given Filter function. The Filter function operates directly on row numbers into the Table as these row numbers have already been projected through the indexes.
func (*IndexView) FilterCol ¶
FilterCol sorts the indexes into our Table according to values in given column index, using string representation of column values. Includes rows with matching values unless exclude is set. If contains, only checks if row contains string; if ignoreCase, ignores case. Use named args for greater clarity. Only valid for 1-dimensional columns.
func (*IndexView) FilterColName ¶
func (ix *IndexView) FilterColName(colNm string, str string, exclude, contains, ignoreCase bool) error
FilterColName filters the indexes into our Table according to values in given column name, using string representation of column values. Includes rows with matching values unless exclude is set. If contains, only checks if row contains string; if ignoreCase, ignores case. Use named args for greater clarity. Only valid for 1-dimensional columns. Returns error if column name not found.
func (*IndexView) InsertRows ¶
InsertRows adds n rows to end of underlying Table, and to the indexes starting at given index in this view
func (*IndexView) NewTable ¶
NewTable returns a new table with column data organized according to the indexes
func (*IndexView) OpenCSV ¶
OpenCSV reads a table idx view from a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg), using the Go standard encoding/csv reader conforming to the official CSV standard. If the table does not currently have any columns, the first row of the file is assumed to be headers, and columns are constructed therefrom. The C++ emergent column headers are parsed -- these have full configuration information for tensor dimensionality. If the table DOES have existing columns, then those are used robustly for whatever information fits from each row of the file.
func (*IndexView) OpenFS ¶
OpenFS is the version of IndexView.OpenCSV that uses an fs.FS filesystem.
func (*IndexView) Permuted ¶
func (ix *IndexView) Permuted()
Permuted sets indexes to a permuted order -- if indexes already exist then existing list of indexes is permuted, otherwise a new set of permuted indexes are generated
func (*IndexView) RowsByString ¶
RowsByString returns the list of *our indexes* whose row in the table has given string value in given column name (de-reference our indexes to get actual row). if contains, only checks if row contains string; if ignoreCase, ignores case. returns nil if name invalid -- see also Try. Use named args for greater clarity.
func (*IndexView) RowsByStringIndex ¶
RowsByStringIndex returns the list of *our indexes* whose row in the table has given string value in given column index (de-reference our indexes to get actual row). if contains, only checks if row contains string; if ignoreCase, ignores case. Use named args for greater clarity.
func (*IndexView) RowsByStringTry ¶
func (ix *IndexView) RowsByStringTry(colNm string, str string, contains, ignoreCase bool) ([]int, error)
RowsByStringTry returns the list of *our indexes* whose row in the table has given string value in given column name (de-reference our indexes to get actual row). if contains, only checks if row contains string; if ignoreCase, ignores case. returns error message for invalid column name. Use named args for greater clarity.
func (*IndexView) SaveCSV ¶
SaveCSV writes a table idx view to a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg). If headers = true then generate C++ emergent-tyle column headers. These headers have full configuration information for the tensor columns. Otherwise, only the data is written.
func (*IndexView) Sequential ¶
func (ix *IndexView) Sequential()
Sequential sets indexes to sequential row-wise indexes into table
func (*IndexView) SetTable ¶
SetTable sets as indexes into given table with sequential initial indexes
func (*IndexView) Sort ¶
Sort sorts the indexes into our Table using given Less function. The Less function operates directly on row numbers into the Table as these row numbers have already been projected through the indexes.
func (*IndexView) SortCol ¶
SortCol sorts the indexes into our Table according to values in given column index, using either ascending or descending order. Only valid for 1-dimensional columns.
func (*IndexView) SortColName ¶
SortColName sorts the indexes into our Table according to values in given column name, using either ascending or descending order. Only valid for 1-dimensional columns. Returns error if column name not found.
func (*IndexView) SortColNames ¶
SortColNames sorts the indexes into our Table according to values in given column names, using either ascending or descending order. Only valid for 1-dimensional columns. Returns error if column name not found.
func (*IndexView) SortCols ¶
SortCols sorts the indexes into our Table according to values in given list of column indexes, using either ascending or descending order for all of the columns. Only valid for 1-dimensional columns.
func (*IndexView) SortIndexes ¶
func (ix *IndexView) SortIndexes()
SortIndexes sorts the indexes into our Table directly in numerical order, producing the native ordering, while preserving any filtering that might have occurred.
func (*IndexView) SortStable ¶
SortStable stably sorts the indexes into our Table using given Less function. The Less function operates directly on row numbers into the Table as these row numbers have already been projected through the indexes. It is *essential* that it always returns false when the two are equal for the stable function to actually work.
func (*IndexView) SortStableCol ¶
SortStableCol sorts the indexes into our Table according to values in given column index, using either ascending or descending order. Only valid for 1-dimensional columns.
func (*IndexView) SortStableColName ¶
SortStableColName sorts the indexes into our Table according to values in given column name, using either ascending or descending order. Only valid for 1-dimensional columns. Returns error if column name not found.
func (*IndexView) SortStableColNames ¶
SortStableColNames sorts the indexes into our Table according to values in given column names, using either ascending or descending order. Only valid for 1-dimensional columns. Returns error if column name not found.
func (*IndexView) SortStableCols ¶
SortStableCols sorts the indexes into our Table according to values in given list of column indexes, using either ascending or descending order for all of the columns. Only valid for 1-dimensional columns.
func (*IndexView) WriteCSV ¶
WriteCSV writes only rows in table idx view to a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg). If headers = true then generate C++ emergent-style column headers. These headers have full configuration information for the tensor columns. Otherwise, only the data is written.
type LessFunc ¶
LessFunc is a function used for sort comparisons that returns true if Table row i is less than Table row j -- these are the raw row numbers, which have already been projected through indexes when used for sorting via Indexes.
type Schema ¶
type Schema []Column
Schema specifies all of the columns of a table, sufficient to create the table. It is just a slice list of Columns
func SchemaFromEmerHeaders ¶
SchemaFromEmerHeaders attempts to configure a Table Schema based on emergent DataTable headers
func SchemaFromHeaders ¶
SchemaFromHeaders attempts to configure a Table Schema based on the headers for non-Emergent headers, data is examined to
type SplitAgg ¶
type SplitAgg struct { // the name of the aggregation operation performed, e.g., Sum, Mean, etc Name string // column index on which the aggregation was performed -- results will have same shape as cells in this column ColIndex int // aggregation results -- outer index is length of splits, inner is the length of the cell shape for the column Aggs [][]float64 }
SplitAgg contains aggregation results for splits
type Splits ¶
type Splits struct { // the list of index views for each split Splits []*IndexView // levels of indexes used to organize the splits -- each split contains the full outer product across these index levels. for example, if the split was generated by grouping over column values, then these are the column names in order of grouping. the splits are not automatically sorted hierarchically by these levels but e.g., the GroupBy method produces that result -- use the Sort methods to explicitly sort. Levels []string // the values of the index levels associated with each split. The outer dimension is the same length as Splits, and the inner dimension is the levels. Values [][]string // aggregate results, one for each aggregation operation performed -- split-level data is contained within each SplitAgg struct -- deleting a split removes these aggs but adding new splits just invalidates all existing aggs (they are automatically deleted). Aggs []*SplitAgg // contains filtered or unexported fields }
Splits is a list of indexed views into a given Table, that represent a particular way of splitting up the data, e.g., whenever a given column value changes.
It is functionally equivalent to the MultiIndex in python's pandas: it has multiple levels of indexes as listed in the Levels field, which then have corresponding Values for each split. These index levels can be re-ordered, and new Splits or IndexViews's can be created from subsets of the existing levels. The Values are stored simply as string values, as this is the most general type and often index values are labels etc.
For Splits created by the splits.GroupBy function for example, each index Level is the column name that the data was grouped by, and the Values for each split are then the values of those columns. However, any arbitrary set of levels and values can be used, e.g., as in the splits.GroupByFunc function.
Conceptually, a given Split always contains the full "outer product" of all the index levels -- there is one split for each unique combination of values along each index level. Thus, removing one level collapses across those values and moves the corresponding indexes into the remaining split indexes.
You can Sort and Filter based on the index values directly, to reorganize the splits and drop particular index values, etc.
Splits also maintains Aggs aggregate values for each split, which can be computed using standard aggregation methods over data columns, using the split.Agg* functions.
The etable code contains the structural methods for managing the Splits data. See split package for end-user methods to generate different kinds of splits, and perform aggregations, etc.
func (*Splits) AggByColName ¶
AggByColName returns Agg results for given column name, optionally including :Name agg name appended, where Name is the name given to the Agg result (e.g., Mean for a standard Mean agg). Returns nil if not found. See also Try version for error message.
func (*Splits) AggByColNameTry ¶
AggByColNameTry returns Agg results for given column name, optionally including :Name agg name appended, where Name is the name given to the Agg result (e.g., Mean for a standard Mean agg). Returns error message if not found.
func (*Splits) AggByName ¶
AggByName returns Agg results for given name, which does NOT include the column name, just the name given to the Agg result (e.g., Mean for a standard Mean agg). See also AggByColName. Returns nil if not found. See also Try version for error message.
func (*Splits) AggByNameTry ¶
AggByNameTry returns Agg results for given name, which does NOT include the column name, just the name given to the Agg result (e.g., Mean for a standard Mean agg). See also AggByColName. Returns error message if not found.
func (*Splits) AggsToTable ¶
AggsToTable returns a Table containing this Splits' aggregate data. Must have Levels and Aggs all created as in the split.Agg* methods. if colName == ColNameOnly, then the name of the columns for the Table is just the corresponding agg column name -- otherwise it also includes the name of the aggregation function with a : divider (e.g., Name:Mean)
func (*Splits) AggsToTableCopy ¶
AggsToTableCopy returns a Table containing this Splits' aggregate data and a copy of the first row of data for each split for all non-agg cols, which is useful for recording other data that goes along with aggregated values. Must have Levels and Aggs all created as in the split.Agg* methods. if colName == ColNameOnly, then the name of the columns for the Table is just the corresponding agg column name -- otherwise it also includes the name of the aggregation function with a : divider (e.g., Name:Mean)
func (*Splits) ByValue ¶
ByValue finds split indexes by matching to split values, returns nil if not found. values are used in order as far as they go and any remaining values are assumed to match, and any empty values will match anything. Can use this to access different subgroups within overall set of splits.
func (*Splits) CopyFrom ¶
CopyFrom copies from other Splits -- we get our own unique copy of everything
func (*Splits) Delete ¶
Delete deletes split at given index -- use this to coordinate deletion of Splits, Values, and Aggs values for given split
func (*Splits) DeleteAggs ¶
func (spl *Splits) DeleteAggs()
DeleteAggs deletes all existing aggregation data
func (*Splits) ExtractLevels ¶
ExtractLevels returns a new Splits that only has the given levels of indexes, in their given order, with the other levels removed and their corresponding indexes merged into the appropriate remaining levels. Any existing aggregation data is not retained in the new splits.
func (*Splits) New ¶
New adds a new split to the list for given table, and with associated values, which are copied before saving into Values list, and any number of rows from the table associated with this split (also copied). Any existing Aggs are deleted by this.
func (*Splits) ReorderLevels ¶
ReorderLevels re-orders the index levels according to the given new ordering indexes e.g., []int{1,0} will move the current level 0 to level 1, and 1 to level 0 no checking is done to ensure these are sensible beyond basic length test -- behavior undefined if so. Typically you want to call SortLevels after this.
func (*Splits) SetLevels ¶
SetLevels sets the Levels index names -- must match actual index dimensionality of the Values. This is automatically done by e.g., GroupBy, but must be done manually if creating custom indexes.
func (*Splits) SortLevels ¶
func (spl *Splits) SortLevels()
SortLevels sorts the splits according to the current index level ordering of values i.e., first index level is outer sort dimension, then within that is the next, etc
type SplitsLessFunc ¶
SplitsLessFunc is a function used for sort comparisons that returns true if split i is less than split j
type Table ¶
type Table struct { // columns of data, as etensor.Tensor tensors Cols []etensor.Tensor `view:"no-inline"` // the names of the columns ColNames []string // number of rows, which is enforced to be the size of the outer-most dimension of the column tensors Rows int `edit:"-"` // the map of column names to column numbers ColNameMap map[string]int `view:"-"` // misc meta data for the table. We use lower-case key names following the struct tag convention: name = name of table; desc = description; read-only = gui is read-only; precision = n for precision to write out floats in csv. For Column-specific data, we look for ColName: prefix, specifically ColName:desc = description of the column contents, which is shown as tooltip in the etview.TableView, and :width for width of a column MetaData map[string]string }
etable.Table is the emer DataTable structure, containing columns of etensor tensors. All tensors MUST have RowMajor stride layout!
func New ¶
New returns a new Table constructed from given Schema. The actual tensor number of rows is enforced to be > 0, because we cannot have a null dimension in tensor shape
func (*Table) AddCol ¶
AddCol adds the given tensor as a column to the table. returns error if it is not a RowMajor organized tensor, and automatically adjusts the shape to fit the current number of rows.
func (*Table) AppendRows ¶
AppendRows appends shared columns in both tables with input table rows
func (*Table) CellFloat ¶
CellFloat returns the float64 value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns NaN if column is not a 1-dimensional tensor or col name not found, or row not valid.
func (*Table) CellFloatIndex ¶
CellFloatIndex returns the float64 value of cell at given column, row index for columns that have 1-dimensional tensors. Returns NaN if column is not a 1-dimensional tensor or row not valid.
func (*Table) CellFloatTry ¶
CellFloatTry returns the float64 value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns an error if column not found, or column is not a 1-dimensional tensor, or row not valid.
func (*Table) CellString ¶
CellString returns the string value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns "" if column is not a 1-dimensional tensor or row not valid.
func (*Table) CellStringIndex ¶
CellStringIndex returns the string value of cell at given column, row index for columns that have 1-dimensional tensors. Returns "" if column is not a 1-dimensional tensor or row not valid.
func (*Table) CellStringTry ¶
CellStringTry returns the string value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns an error if column not found, or column is not a 1-dimensional tensor, or row not valid.
func (*Table) CellTensor ¶
CellTensor returns the tensor SubSpace for given column (by name), row index for columns that have higher-dimensional tensors so each row is represented by an n-1 dimensional tensor, with the outer dimension being the row number. Returns nil on any error -- see Try version for error returns.
func (*Table) CellTensorFloat1D ¶
CellTensorFloat1D returns the float value of a Tensor cell's cell at given 1D offset within cell, for given column (by name), row index for columns that have higher-dimensional tensors so each row is represented by an n-1 dimensional tensor, with the outer dimension being the row number. Returns 0 on any error -- see Try version for error returns.
func (*Table) CellTensorFloat1DTry ¶
CellTensorFloat1DTry returns the float value of a Tensor cell's cell at given 1D offset within cell, for given column (by name), row index for columns that have higher-dimensional tensors so each row is represented by an n-1 dimensional tensor, with the outer dimension being the row number. Returns any error.
func (*Table) CellTensorIndex ¶
CellTensorIndex returns the tensor SubSpace for given column, row index for columns that have higher-dimensional tensors so each row is represented by an n-1 dimensional tensor, with the outer dimension being the row number. Returns nil if column is a 1-dimensional tensor or there is any error from the etensor.Tensor.SubSpace call.
func (*Table) CellTensorTry ¶
CellTensorTry returns the tensor SubSpace for given column (by name), row index for columns that have higher-dimensional tensors so each row is represented by an n-1 dimensional tensor, with the outer dimension being the row number. Returns an error if column is a 1-dimensional tensor or any error from the etensor.Tensor.SubSpace call.
func (*Table) ColByName ¶
ColByName returns the tensor at given column name without any error messages -- just returns nil if not found
func (*Table) ColByNameTry ¶
ColByNameTry returns the tensor at given column name, if not found, returns error
func (*Table) ColIndex ¶
ColIndex returns the index of the given column name. returns -1 if name not found -- see Try version for error message.
func (*Table) ColIndexTry ¶
ColIndexTry returns the index of the given column name, along with an error if not found.
func (*Table) ColIndexesByNames ¶
ColIndexesByNames returns the indexes of the given column names. idxs have -1 if name not found -- see Try version for error message.
func (*Table) ColIndexesByNamesTry ¶
ColsIndexesByNamesTry returns the indexes of the given column names, along with an error if any not found.
func (*Table) CopyCell ¶
CopyCell copies into cell at given col, row from cell in other table. It is robust to differences in type -- uses destination cell type. Returns error if column names are invalid.
func (*Table) CopyMetaDataFrom ¶
CopyMetaDataFrom copies meta data from other table
func (*Table) DeleteColIndex ¶
DeleteColIndex deletes column of given index
func (*Table) DeleteColName ¶
DeleteColName deletes column of given name.
func (*Table) EmerHeaders ¶
EmerHeaders generates emergent DataTable header strings from the table. These have full information about type and tensor cell dimensionality.
func (*Table) IsValidRow ¶
IsValidRow returns true if the row is valid
func (*Table) IsValidRowTry ¶
IsValidRowTry returns an error message if the row is not valid.
func (*Table) OpenCSV ¶
OpenCSV reads a table from a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg), using the Go standard encoding/csv reader conforming to the official CSV standard. If the table does not currently have any columns, the first row of the file is assumed to be headers, and columns are constructed therefrom. The C++ emergent column headers are parsed -- these have full configuration information for tensor dimensionality. If the table DOES have existing columns, then those are used robustly for whatever information fits from each row of the file.
func (*Table) OpenFS ¶
OpenFS is the version of Table.OpenCSV that uses an fs.FS filesystem.
func (*Table) ReadCSV ¶
ReadCSV reads a table from a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg), using the Go standard encoding/csv reader conforming to the official CSV standard. If the table does not currently have any columns, the first row of the file is assumed to be headers, and columns are constructed therefrom. The C++ emergent column headers are parsed -- these have full configuration information for tensor dimensionality. If the table DOES have existing columns, then those are used robustly for whatever information fits from each row of the file.
func (*Table) ReadCSVRow ¶
ReadCSVRow reads a record of CSV data into given row in table
func (*Table) RowsByString ¶
RowsByString returns the list of rows that have given string value in given column name. returns nil if name invalid -- see also Try. if contains, only checks if row contains string; if ignoreCase, ignores case. Use named args for greater clarity.
func (*Table) RowsByStringIndex ¶
RowsByStringIndex returns the list of rows that have given string value in given column index. if contains, only checks if row contains string; if ignoreCase, ignores case. Use named args for greater clarity.
func (*Table) RowsByStringTry ¶
func (dt *Table) RowsByStringTry(colNm string, str string, contains, ignoreCase bool) ([]int, error)
RowsByStringTry returns the list of rows that have given string value in given column name. returns error message for invalid column name. if contains, only checks if row contains string; if ignoreCase, ignores case. Use named args for greater clarity.
func (*Table) SaveCSV ¶
SaveCSV writes a table to a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg). If headers = true then generate C++ emergent-tyle column headers. These headers have full configuration information for the tensor columns. Otherwise, only the data is written.
func (*Table) SetCellFloat ¶
SetCellFloat sets the float64 value of cell at given column (by name), row index for columns that have 1-dimensional tensors.
func (*Table) SetCellFloatIndex ¶
SetCellFloatIndex sets the float64 value of cell at given column, row index for columns that have 1-dimensional tensors. Returns true if set.
func (*Table) SetCellFloatTry ¶
SetCellFloatTry sets the float64 value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns an error if column not found, or column is not a 1-dimensional tensor.
func (*Table) SetCellString ¶
SetCellString sets the string value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns true if set.
func (*Table) SetCellStringIndex ¶
SetCellStringIndex sets the string value of cell at given column, row index for columns that have 1-dimensional tensors. Returns true if set.
func (*Table) SetCellStringTry ¶
SetCellStringTry sets the string value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns an error if column not found, or column is not a 1-dimensional tensor.
func (*Table) SetCellTensor ¶
SetCellTensor sets the tensor value of cell at given column (by name), row index for columns that have n-dimensional tensors. Returns true if set.
func (*Table) SetCellTensorFloat1D ¶
SetCellTensorFloat1D sets the tensor cell's float cell value at given 1D index within cell, at given column (by name), row index for columns that have n-dimensional tensors. Returns true if set.
func (*Table) SetCellTensorFloat1DTry ¶
SetCellTensorFloat1DTry sets the string value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns an error if column not found, or column is not a 1-dimensional tensor.
func (*Table) SetCellTensorIndex ¶
SetCellTensorIndex sets the tensor value of cell at given column, row index for columns that have n-dimensional tensors. Returns true if set.
func (*Table) SetCellTensorTry ¶
SetCellTensorTry sets the string value of cell at given column (by name), row index for columns that have 1-dimensional tensors. Returns an error if column not found, or column is not a 1-dimensional tensor.
func (*Table) SetFromSchema ¶
SetFromSchema configures table from given Schema. The actual tensor number of rows is enforced to be > 0, because we cannot have a null dimension in tensor shape. does not preserve any existing columns / data.
func (*Table) SetMetaData ¶
SetMetaData sets given meta-data key to given value, safely creating the map if not yet initialized. Standard Keys are: * name -- name of table * desc -- description of table * read-only -- makes gui read-only (inactive edits) for etview.TableView * ColName:* -- prefix for all column-specific meta-data
- desc -- description of column
func (*Table) SetNumRows ¶
SetNumRows sets the number of rows in the table, across all columns if rows = 0 then effective number of rows in tensors is 1, as this dim cannot be 0
func (*Table) UpdateColNameMap ¶
func (dt *Table) UpdateColNameMap()
UpdateColNameMap updates the column name map
func (*Table) WriteCSV ¶
WriteCSV writes a table to a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg). If headers = true then generate C++ emergent-style column headers. These headers have full configuration information for the tensor columns. Otherwise, only the data is written.
func (*Table) WriteCSVHeaders ¶
WriteCSVHeaders writes headers to a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg). Returns number of columns in header
func (*Table) WriteCSVRow ¶
WriteCSVRow writes given row to a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg)