ds

Documentation

Overview

Package ds exposes the qri dataset document model into starlark

Package ds defines the qri dataset object within starlark

outline: ds
  ds defines the qri dataset object within starlark. it's loaded by default
  in the qri runtime

  types:
    Dataset
      a qri dataset. Datasets can be either read-only or read-write. By default datasets are read-write
      methods:
        set_meta(meta dict)
          set dataset meta component
        get_meta() dict|None
          get dataset meta component
        get_structure() dict|None
          get dataset structure component if one is defined
        set_structure(structure) structure
          set dataset structure component
        get_body() dict|list|None
          get dataset body component if one is defined
        set_body(data dict|list, parse_as? string) body
          set dataset body component. set_body has only one optional argument: 'parse_as', which defaults to the
          empty string. By default qri assumes the data value provided to set_body is an iterable starlark data
          structure (tuple, set, list, dict). When parse_as is set, set_body assumes the provided body value will
          be a string of serialized structured data in the given format. valid parse_as values are "json", "csv",
          "cbor", "xlsx".

Index

• Constants
• func LoadModule() (starlark.StringDict, error)
• func New(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, ...) (starlark.Value, error)
• type Dataset
  • func NewDataset(ds *dataset.Dataset, check MutateFieldCheck) *Dataset
  • func (d *Dataset) GetBody(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, ...) (starlark.Value, error)
  • func (d *Dataset) GetMeta(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, ...) (starlark.Value, error)
  • func (d *Dataset) GetStructure(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, ...) (starlark.Value, error)
  • func (d *Dataset) IsBodyModified() bool
  • func (d *Dataset) Methods() *starlarkstruct.Struct
  • func (d *Dataset) SetBody(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, ...) (starlark.Value, error)
  • func (d *Dataset) SetMeta(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, ...) (starlark.Value, error)
  • func (d *Dataset) SetMutable(ds *dataset.Dataset)
  • func (d *Dataset) SetStructure(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, ...) (starlark.Value, error)
• type EntryReader
  • func NewEntryReader(st *dataset.Structure, iter starlark.Iterable) *EntryReader
  • func (r *EntryReader) Close() error
  • func (r *EntryReader) ReadEntry() (e dsio.Entry, err error)
  • func (r *EntryReader) Structure() *dataset.Structure
• type MutateFieldCheck
• type StarlarkEntryWriter
  • func NewStarlarkEntryWriter(st *dataset.Structure) (*StarlarkEntryWriter, error)
  • func (w *StarlarkEntryWriter) Close() error
  • func (w *StarlarkEntryWriter) Structure() *dataset.Structure
  • func (w *StarlarkEntryWriter) Value() starlark.Value
  • func (w *StarlarkEntryWriter) WriteEntry(ent dsio.Entry) error

Constants

const ModuleName = "dataset.star"

ModuleName defines the expected name for this Module when used in starlark's load() function, eg: load('dataset.star', 'dataset')

Functions

func LoadModule

func LoadModule() (starlark.StringDict, error)

LoadModule loads the base64 module. It is concurrency-safe and idempotent.

func New

func New(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error)

New creates a new dataset from starlark land

Types

type Dataset

type Dataset struct {
	// contains filtered or unexported fields
}

Dataset is a qri dataset starlark type

func NewDataset

func NewDataset(ds *dataset.Dataset, check MutateFieldCheck) *Dataset

NewDataset creates a dataset object, intended to be called from go-land to prepare datasets for handing to other functions

func (*Dataset) GetBody

func (d *Dataset) GetBody(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error)

GetBody returns the body of the dataset we're transforming. The read version is returned until the dataset is modified by set_body, then the write version is returned instead.

func (*Dataset) GetMeta

func (d *Dataset) GetMeta(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error)

GetMeta gets a dataset meta component

func (*Dataset) GetStructure

func (d *Dataset) GetStructure(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error)

GetStructure gets a dataset structure component

func (*Dataset) IsBodyModified

func (d *Dataset) IsBodyModified() bool

IsBodyModified returns whether the body has been modified by set_body

func (*Dataset) Methods

func (d *Dataset) Methods() *starlarkstruct.Struct

Methods exposes dataset methods as starlark values

func (*Dataset) SetBody

func (d *Dataset) SetBody(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error)

SetBody assigns the dataset body. Future calls to GetBody will return this newly mutated body, even if assigned value is the same as what was already there.

func (*Dataset) SetMeta

func (d *Dataset) SetMeta(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error)

SetMeta sets a dataset meta field

func (*Dataset) SetMutable

func (d *Dataset) SetMutable(ds *dataset.Dataset)

SetMutable assigns an underlying dataset that can be mutated

func (*Dataset) SetStructure

func (d *Dataset) SetStructure(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error)

SetStructure sets the dataset structure component

type EntryReader

type EntryReader struct {
	// contains filtered or unexported fields
}

EntryReader implements the dsio.EntryReader interface for starlark.Iterable's

func NewEntryReader

func NewEntryReader(st *dataset.Structure, iter starlark.Iterable) *EntryReader

NewEntryReader creates a new Entry Reader

func (*EntryReader) Close

func (r *EntryReader) Close() error

Close finalizes the reader

func (*EntryReader) ReadEntry

func (r *EntryReader) ReadEntry() (e dsio.Entry, err error)

ReadEntry reads one entry from the reader

func (*EntryReader) Structure

func (r *EntryReader) Structure() *dataset.Structure

Structure gives this reader's structure

type MutateFieldCheck

type MutateFieldCheck func(path ...string) error

MutateFieldCheck is a function to check if a dataset field can be mutated before mutating a field, dataset will call MutateFieldCheck with as specific a path as possible and bail if an error is returned

type StarlarkEntryWriter

type StarlarkEntryWriter struct {
	IsDict bool
	Struct *dataset.Structure
	Object starlark.Value
}

StarlarkEntryWriter creates a starlark.Value as an EntryWriter

func NewStarlarkEntryWriter

func NewStarlarkEntryWriter(st *dataset.Structure) (*StarlarkEntryWriter, error)

NewStarlarkEntryWriter returns a new StarlarkEntryWriter

func (*StarlarkEntryWriter) Close

func (w *StarlarkEntryWriter) Close() error

Close is a no-op

func (*StarlarkEntryWriter) Structure

func (w *StarlarkEntryWriter) Structure() *dataset.Structure

Structure returns the EntryWriter's dataset structure

func (*StarlarkEntryWriter) Value

func (w *StarlarkEntryWriter) Value() starlark.Value

Value returns the underlying starlark.Value

func (*StarlarkEntryWriter) WriteEntry

func (w *StarlarkEntryWriter) WriteEntry(ent dsio.Entry) error

WriteEntry adds an entry to the underlying starlark.Value